ImportError: cannot import name error replace_entities when using Scrapy

In the past I have created a spider using scrapy, I write scrapy startproject some_project

I recently cloned a spider repository and now when I go to the correct location and type scrapy crawl some_spider -o output.csv -t csv

I get import errors:

    Traceback (most recent call last):
  File "/usr/local/bin/scrapy", line 3, in <module>
    from scrapy.cmdline import execute
  File "/usr/lib/pymodules/python2.7/scrapy/", line 58, in <module>
    from scrapy.selector import Selector
  File "/usr/lib/pymodules/python2.7/scrapy/selector/", line 4, in <module>
    from scrapy.selector.unified import *
  File "/usr/lib/pymodules/python2.7/scrapy/selector/", line 7, in <module>
    from scrapy.utils.misc import extract_regex
  File "/usr/lib/pymodules/python2.7/scrapy/utils/", line 8, in <module>
    from w3lib.html import replace_entities
ImportError: cannot import name replace_entities


I googled around and tried to see what is happening with `replace_entities' but I cannot find any information. Any help as to why these import errors are occurring, and any ideas on how to fix this would be greatly appreciated.


source to share

1 answer


- dependency Scrapy

, quote from

(version 0.24.4):



As you can see, it Scrapy

requires w3lib

version 1.8.0 or higher.

The solution is to update the package w3lib


pip install --upgrade w3lib 




All Articles