A messy word when using scrapy to save json data to csv

I have used scrapy to crawl comments from a website http://club.jd.com/comment/productPageComments.action?callback=&productId=1892018&score=0&sortType=5&page=0&pageSize=10&isShadowSku=0

that is Chinese. But I just got the result like this:enter image description here

And the csv file output is all messed up. enter image description here

I don't know what happened. At first I thought it was a problem json decode

or json encode

, and then I tried ways on the internet, but I got the same result. Here's my code:

#!/usr/bin/env python
# encoding: utf-8

import scrapy
from scrapy import Request
from scrapy.selector import Selector
from jd_comment.items import JdCommentItem
import json

class JdSpider(scrapy.Spider):
    name = 'comment'

    def start_requests(self):
        url = 'http://club.jd.com/comment/productPageComments.action?callback=&productId=1892018&score=0&sortType=5&page=0&pageSize=10&isShadowSku=0'
        yield scrapy.Request(url=url, callback=self.parse)

    def parse(self, response):
        jsonresponse = json.loads(response.body_as_unicode())
        items = []
        for comment in jsonresponse['comments']:
            item = JdCommentItem()
            item['username'] = comment['nickname']
            item['user_ID'] = comment['id']
            item['time'] = comment['referenceTime']
            item['good_ID'] = comment['referenceId']
            item['good_name'] = comment['referenceName']
            item['content'] = comment['content']
            item['score'] = comment['score']

            items.append(item)

            yield item

      

Anyone who gives me a hint would be much appreciated. Thank.

+3
json python csv


source to share


No one has answered this question yet

Check out similar questions:

2480
How do I send JSON data using Curl from terminal / command line to Test Spring REST?
928
How do I write JSON data to a file?
774
Save PL / pgSQL output from PostgreSQL to CSV file
658
What is the difference between YAML and JSON?
615
Posting the file and related data to a RESTful WebService preferably in JSON
574
Binary data in JSON string. Something Better Than Base64
534
How to import CSV files into PostgreSQL table?
1
Unicode on Scrapy Json output
0
Scrapy - Error Importing JSON
0
scrapy adds data to json file from different functions



All Articles
Loading...
X
Show
Funny
Dev
Pics