ImportError: cannot import name error replace_entities when using Scrapy

In the past I have created a spider using scrapy, I write scrapy startproject some_project

I recently cloned a spider repository and now when I go to the correct location and type scrapy crawl some_spider -o output.csv -t csv


I get import errors:

    Traceback (most recent call last):
  File "/usr/local/bin/scrapy", line 3, in <module>
    from scrapy.cmdline import execute
  File "/usr/lib/pymodules/python2.7/scrapy/__init__.py", line 58, in <module>
    from scrapy.selector import Selector
  File "/usr/lib/pymodules/python2.7/scrapy/selector/__init__.py", line 4, in <module>
    from scrapy.selector.unified import *
  File "/usr/lib/pymodules/python2.7/scrapy/selector/unified.py", line 7, in <module>
    from scrapy.utils.misc import extract_regex
  File "/usr/lib/pymodules/python2.7/scrapy/utils/misc.py", line 8, in <module>
    from w3lib.html import replace_entities
ImportError: cannot import name replace_entities

      

I googled around and tried to see what is happening with `replace_entities' but I cannot find any information. Any help as to why these import errors are occurring, and any ideas on how to fix this would be greatly appreciated.

+3


source to share


1 answer


w3lib

- dependency Scrapy

, quote from setup.py

(version 0.24.4):

install_requires=[
    'Twisted>=10.0.0',
    'w3lib>=1.8.0',
    'queuelib',
    'lxml',
    'pyOpenSSL',
    'cssselect>=0.9',
    'six>=1.5.2',
],

      

As you can see, it Scrapy

requires w3lib

version 1.8.0 or higher.



The solution is to update the package w3lib

:

pip install --upgrade w3lib 

      

+2


source







All Articles