Haystack UnicodeDecodeError on recovery index

Question

Haystack UnicodeDecodeError on recovery index

I am using Haystack with Solr on a small project and I have a problem. The problem seems to be related to Unicode characters, I have a model called Channel and one of the fields is description. This field contains non-ascii characters.

class Channel(models.Model):
    owner = models.ForeignKey(User, null=True, related_name='channels')
    name = models.CharField(max_length=20, db_index=True, unique=True)
    image = models.URLField()
    description = models.CharField(max_length=255)
    kind = models.IntegerField(default=1, choices=CHANNEL_CHOICES)
    hidden = models.BooleanField(default=False)

    subscriptions = models.IntegerField(default=0)

    date_created = models.DateTimeField(auto_now_add=True)
    date_modified = models.DateTimeField(auto_now=True)

I did SearchIndex like this

class ChannelIndex(indexes.SearchIndex, indexes.Indexable):
    text = indexes.CharField(document=True, use_template=True)

    name = indexes.NgramField(model_attr='name')
    id = indexes.IntegerField(model_attr='id')
    description = indexes.CharField(model_attr='description')
    image = indexes.CharField(model_attr='image')
    hidden = indexes.BooleanField(model_attr='hidden')
    kind = indexes.IntegerField(model_attr='kind')

    def get_model(self):
        return Channel

Now when there are ascii characters in the description field (but I think the same can happen with others), I get an error trying to rebuild the index.

  File "/usr/lib/python2.7/httplib.py", line 962, in request
self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 996, in _send_request
self.endheaders(body)
  File "/usr/lib/python2.7/httplib.py", line 958, in endheaders
self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 816, in _send_output
msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 5173: ordinal not in range(128)