Clever Geek Handbook

How to prevent python urllib3 from caching response

I am processing the same url multiple times in a row with urllib3. I have profiled the requests and I get the header:

Cache-Control: no-transform, max-age = 120

On requests after the first one, I get the cached version of the page, not the request that runs again. I cannot control the headers returned from the server, how can I prevent caching?

I am on Debian Squeeze and it is running python 2.6.6

pool = urllib3.HTTPConnectionPool('itunes.apple.com')
request = pool.request('GET', '/webObjects/MZStore.woa/wa/viewTop?
                       selected_tab_index=0&startIndex=0&genreId=36', 
                       headers = {'Host': 'itunes.apple.com', 
                       'Accept-Encoding': 'gzip, deflate', 'X-Apple-Tz': -18000, 
                       'X-Apple-Store-Front': '143441-1,2')
+3


source to share


2 answers


Urllib3 does not have built-in caching. Are you sure the server is not responding to cached results?



+3


source


Try adding the following headers:

Cache-Control: no-cache



Pragma: no-cache

Expires: Thu, 01 Jan 1970 00:00:00 GMT

+2


source






More articles:


All Articles