Shelf module problem?
Using the shelf module gave me amazing behavior. keys (), iter () and iteritems () do not return all shelf entries! Here's the code:
cache = shelve.open('my.cache')
# ...
cache[url] = (datetime.datetime.today(), value)
later:
cache = shelve.open('my.cache')
urls = ['accounts_with_transactions.xml', 'targets.xml', 'profile.xml']
try:
print list(cache.keys()) # doesn't return all the keys!
print [url for url in urls if cache.has_key(url)]
print list(cache.keys())
finally:
cache.close()
and here's the output:
['targets.xml'] ['accounts_with_transactions.xml', 'targets.xml'] ['targets.xml', 'accounts_with_transactions.xml']
Has anyone encountered this before, and is there a workaround without knowing all the possible cache keys a priori?
+2
source to share
2 answers
As per the python library link :
... The database is also (unfortunately) subject to dbm restrictions if used - which means the (pickled view) of the objects stored in the database must be pretty small ...
This correctly reproduces the "error":
import shelve
a = 'trxns.xml'
b = 'foobar.xml'
c = 'profile.xml'
urls = [a, b, c]
cache = shelve.open('my.cache', 'c')
try:
cache[a] = a*1000
cache[b] = b*10000
finally:
cache.close()
cache = shelve.open('my.cache', 'c')
try:
print cache.keys()
print [url for url in urls if cache.has_key(url)]
print cache.keys()
finally:
cache.close()
with output:
[] ['trxns.xml', 'foobar.xml'] ['foobar.xml', 'trxns.xml']
So the answer doesn't store anything like raw-xml, but rather the results of calculations on the shelf.
+3
source to share