Does anyone know where there is a recipe for serializing data and preserving its order in the output?
I am working with a dataset that I have converted to a list of dictionaries
For example one item in my list
{'reportDate': u'R20070501', 'idnum': u'1078099', 'columnLabel': u'2005',
'actionDate': u'C20070627', 'data': u'76,000', 'rowLabel': u'Sales of Bananas'}
Per request
The second item in my list could be:
{'reportDate': u'R20070501', 'idnum': u'1078099', 'columnLabel': u'2006',
'actionDate': u'C20070627', 'data': u'86,000', 'rowLabel': u'Sales of Bananas'}
The third element could be:
{'reportDate': u'R20070501', 'idnum': u'1078100', 'columnLabel': u'Full Year 2005',
'actionDate': u'C20070627', 'data': u'116,000', 'rowLabel': u'Sales of Cherries'}
The fourth element could be:
{'reportDate': u'R20070501', 'idnum': u'1078100', 'columnLabel': u'Full Year 2006',
'actionDate': u'C20070627', 'data': u'76,000', 'rowLabel': u'Sales of Sales of Cherries'}
The reason I need to sort this is because I need to figure out all the ways to tag columns before I consolidate the results and put them into the database. The first and second items will be one row in the results, the third and fourth will be the next row in the results (after someone decides that there should be the same column header label)
I tested with pickle and was able to save and retrieve my data. However, I need to preserve order in the output. One of my ideas is to add another key, which will be a counter, so that I can get my data and then sort by counter. Is there a better way?
I don't want to put this in the database because it is not persistent.
I have noted the answer below. This is not what I am getting, so I need to figure out if the problem is elsewhere in my code.
source to share
So what about the pickle? If you structure your data as a list of dicts then everything should work the way you want (if I understand your problem).
>>> import pickle
>>> d1 = {1:'one', 2:'two', 3:'three'}
>>> d2 = {1:'eleven', 2:'twelve', 3:'thirteen'}
>>> d3 = {1:'twenty-one', 2:'twenty-two', 3:'twenty-three'}
>>> data = [d1, d2, d3]
>>> out = open('data.pickle', 'wb')
>>> pickle.dump(data, out)
>>> out.close()
>>> input = open('data.pickle')
>>> data2 = pickle.load(input)
>>> data == data2
True
source to share
Python does not preserve order in dictionaries.
However, there is an OrderedDict class in the collections module .
Another option is to use a list of tuples:
[('reportDate', u'R20080501'), ('idnum', u'1078099'), ...etc]
You can use the built-in dict()
if you need to convert it to a dictionary later.
source to share
Python dict is an unordered container. If you need to preserve the order of the entries, you should use a 2-tuple list.
Another option is to keep an additional, ordered list of keys. This way, you can take advantage of the fast accessible key available for the dictionary, but still be able to iterate through its values in an ordered manner:
data = {'reportDate': u'R20070501', 'idnum': u'1078099',
'columnLabel': u'2005', 'actionDate': u'C20070627',
'data': u'76,000', 'rowLabel': u'Sales of Bananas'}
dataOrder = ['reportDate', 'idnum', 'columnLabel',
'actionDate', 'data', 'rowLabel']
for key in dataOrder:
print key, data[key]
source to share