Clever Geek Handbook

Incomplete filedownloads with python

I have the following python code:

import urllib2
DIR = '/home/aaron/Desktop/aaron-file/media/'
LOC = DIR+'/'+bounty.title+'.mp3'

u = urllib2.urlopen(url, 'rb')
localFile = open(LOC, 'wb')
localFile.write(u.read())
localFile.close()
u.close()

It only creates small (about 60K of files) files that work correctly to stop suddenly. When I boot from firefox (copy and paste the same url) I get full size files (about 2MB).

I am running 32 bit ubuntu.

UPDATE: I believe this can be a problem when the length of the http content is inaccurate. How then will I ignore / set different lengths.

Thank.

+3


source to share


1 answer


Perhaps the server you are downloading from is rejecting the default urllib2 user agent. Consider creating a custom opener with a fake user title like:

opener = urllib2.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0')]


and use opener()

insteadurllib2.urlopen()

+1


source






More articles:


All Articles