A messy word when using scrapy to save json data to csv

I have used scrapy to crawl comments from a website http://club.jd.com/comment/productPageComments.action?callback=&productId=1892018&score=0&sortType=5&page=0&pageSize=10&isShadowSku=0

that is Chinese. But I just got the result like this:enter image description here

And the csv file output is all messed up. enter image description here

I don't know what happened. At first I thought it was a problem json decode

or json encode

, and then I tried ways on the internet, but I got the same result. Here's my code:

#!/usr/bin/env python
# encoding: utf-8

import scrapy
from scrapy import Request
from scrapy.selector import Selector
from jd_comment.items import JdCommentItem
import json

class JdSpider(scrapy.Spider):
    name = 'comment'

    def start_requests(self):
        url = 'http://club.jd.com/comment/productPageComments.action?callback=&productId=1892018&score=0&sortType=5&page=0&pageSize=10&isShadowSku=0'
        yield scrapy.Request(url=url, callback=self.parse)

    def parse(self, response):
        jsonresponse = json.loads(response.body_as_unicode())
        items = []
        for comment in jsonresponse['comments']:
            item = JdCommentItem()
            item['username'] = comment['nickname']
            item['user_ID'] = comment['id']
            item['time'] = comment['referenceTime']
            item['good_ID'] = comment['referenceId']
            item['good_name'] = comment['referenceName']
            item['content'] = comment['content']
            item['score'] = comment['score']

            items.append(item)

            yield item

      

Anyone who gives me a hint would be much appreciated. Thank.

+3


source to share





All Articles