Clever Geek Handbook

Get "ParseError" With Scrapy

Most of the time the scraper works fine, but sometimes I get

2014-08-28 11:02:27-0700 [aqicn] DEBUG: Retrying <GET http://site_address_com/> (failed 1 times): [<twisted.python.failure.Failure <class 'twisted.web._newclient.ParseError'>>]

I can't figure out why this is happening. Who has the same problem? Could it be a twisted bug? I am trying to find something, but I only have partial official documentation.

+3


source to share


2 answers


As @nramirezuy and @ B.Mr.W thought, the scraper was banned from the site. I am using TOR and now I am not getting this error. Thanks guys for the help.



+1


source


I think you are using this error . As you can see, there is no official fix right now, just a monkey patch for the ticket.



+1


source






More articles:


All Articles