ImportError: cannot import name error replace_entities when using Scrapy
In the past I have created a spider using scrapy, I write
scrapy startproject some_project
I recently cloned a spider repository and now when I go to the correct location and type
scrapy crawl some_spider -o output.csv -t csv
I get import errors:
Traceback (most recent call last):
File "/usr/local/bin/scrapy", line 3, in <module>
from scrapy.cmdline import execute
File "/usr/lib/pymodules/python2.7/scrapy/__init__.py", line 58, in <module>
from scrapy.selector import Selector
File "/usr/lib/pymodules/python2.7/scrapy/selector/__init__.py", line 4, in <module>
from scrapy.selector.unified import *
File "/usr/lib/pymodules/python2.7/scrapy/selector/unified.py", line 7, in <module>
from scrapy.utils.misc import extract_regex
File "/usr/lib/pymodules/python2.7/scrapy/utils/misc.py", line 8, in <module>
from w3lib.html import replace_entities
ImportError: cannot import name replace_entities
I googled around and tried to see what is happening with `replace_entities' but I cannot find any information. Any help as to why these import errors are occurring, and any ideas on how to fix this would be greatly appreciated.
source to share
w3lib
- dependency Scrapy
, quote from setup.py
(version 0.24.4):
install_requires=[
'Twisted>=10.0.0',
'w3lib>=1.8.0',
'queuelib',
'lxml',
'pyOpenSSL',
'cssselect>=0.9',
'six>=1.5.2',
],
As you can see, it Scrapy
requires w3lib
version 1.8.0 or higher.
The solution is to update the package w3lib
:
pip install --upgrade w3lib
source to share