How to find the difference between two lists of dictionaries checking a key-value pair
I have already looked for a solution to my problem, but with no success. Part of the solution for my problem is here , but that doesn't solve everything.
I have two lists of dictionaries like this - each dictionary is written to a csv file, but I am reading the contents for the following variables:
list1 = [{a:1, b:2, c:3}, {a:4, b:5, c:6}, {a:7, b:8, c:9}]
list2 = [{b:2, a:1, c:3}, {c:6, b:5, a:4}, {b:8, a:7, c:9}]
Using the solution from the link above, that is:
>>> import itertools
>>> a = [{'a': '1'}, {'c': '2'}]
>>> b = [{'a': '1'}, {'b': '2'}]
>>> intersec = [item for item in a if item in b]
>>> sym_diff = [item for item in itertools.chain(a,b) if item not in intersec]
I don't get any matches because the dictionary order is different. But in reality, both lists are the same. How can I check this? Do I have to sort the dictionaries before writing them to the csv file? Could this be the solution?
This is my main problem at the moment, but I have another problem. It would be great if you could check this match, but ignoring one or more keys defined by me. Is it possible?
EDIT: I have dicitonaries in a csv file and I am reading them with the following code:
def read_csv_file(self, filename):
'''Read CSV file and return its content as a Python list.'''
f = open(filename, 'r')
csvfile = csv.reader(f)
f.close
return [row for row in csvfile]
This is very important because I think the problem is that after reading the values from the csv, they are not dictionaries, so the order should be the same.
EDIT2: sample csv file (3 lines, it creates an empty line, but that's not a problem ...)
"{u'Deletion': '0', u'Source': 'Not Applicable', u'Status': ''}"
"{u'Deletion': '0', u'Source': 'Not Applicable', u'Status': ''}"
source to share
Part of this solution was found by the OP according to our last CHAT conversation, it was converting a string to a dictionary using the ast module.
Now, using this module to transform every line read csv.reader()
when it returns a list of lines, which would be a list of one line in the case of a CVS file, add this dictionary to the list. After that, using a list comprehension with help itertools.chain
, we can get the difference between the two lists.
import csv
import ast
import itertools
def csvToList(myCSVFile):
'''This function is used to convert strings returned by csv.reader() into List of dictionaries'''
f = open(myCSVFile, 'r')
l = []
try:
reader = csv.reader(f)
for row in reader:
if row: #as you mentioned in your 2nd edit that you could have empty rows.
l.append(ast.literal_eval(row[0]))
finally:
f.close()
return l
list1 = csvToList('myCSV1.csv')
list2 = csvToList('myCSV2.csv')
l1_sub_l2 = [d for d in list1 if d not in list2]
l2_sub_l1 = [d for d in list2 if d not in list1]
list_difference = list(itertools.chain(l1_sub_l2, l2_sub_l1))
source to share
You need to double check your code. I am not getting the problem you are raising.
list1 = [{a:1, b:2, c:3}, {a:4, b:5, c:6}, {a:7, b:8, c:9}]
list2 = [{b:2, a:1, c:3}, {c:6, b:5, a:4}, {b:8, a:7, c:9}]
list1 = [{'a':1, 'b':2, 'c':3}, {'a':4, 'b':5, 'c':7}, {'a':7, 'b':8, 'c':9}]
list2 = [{'b':2, 'a':1, 'c':3}, {'c':6, 'b':2, 'a':4}, {'b':8, 'a':7, 'c':9}]
intersec = [item for item in list1 if item in list2]
sym_diff = [item for item in itertools.chain(list1,list2) if item not in intersec]
print(intersec)
print(sym_diff)
>>>[{'a': 1, 'c': 3, 'b': 2}, {'a': 4, 'c': 6, 'b': 5}, {'a': 7, 'c': 9, 'b': 8}]
>>>>[]
If I change list1 and list 2 (middle dictionary):
list1 = [{'a':1, 'b':2, 'c':3}, {'a':7, 'b':5, 'c':2}, {'a':7, 'b':8, 'c':9}]
list2 = [{'b':2, 'a':1, 'c':3}, {'c':6, 'b':5, 'a':4}, {'b':8, 'a':7, 'c':9}]
Running the same code:
[{'a': 1, 'c': 3, 'b': 2}, {'a': 7, 'c': 9, 'b': 8}]
[{'a': 7, 'c': 2, 'b': 5}, {'a': 4, 'c': 6, 'b': 5}]
The code provided in the link seems to work fine. Dictionary or list order doesn't matter in python.
source to share