Clever Geek Handbook

Ruby `encode ':" \ xC3 "ASCII-8BIT to UTF-8 (Encoding :: UndefinedConversionError)

The Hannibal episodes in tvdb have strange symbols in them.

For example:

Œuf

So the ruby ​​spits out:

./manifesto.rb:19:in `encode': "\xC3" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)
    from ./manifesto.rb:19:in `to_json'
    from ./manifesto.rb:19:in `<main>'

Line 19:

puts @tree.to_json

Is there a way to deal with these non-utf characters? I would rather convert them instead of replacing them? Or ignore them? I don't know, any help was appreciated.

The weird part is that the script runs fine via cron. An error is generated on manual launch.

+25


source to share


3 answers


It seems you should be using a different encoding for the object. You have to set the correct code page for the variable @tree

, for example using ascii-8bitusing @tree.force_encoding('ISO-8859-1')

. Since it ASCII-8BIT

is only used for binaries.

To find the current external encoding for ruby, run:

Encoding.default_external


If a sudo solves the problem, the problem was the default encoding (encoding), so to fix it, you need to set the correct default encoding (encoding): either

  • In ruby ​​change encoding to utf-8 or another correct one, follow these steps:

    Encoding.default_external = Encoding::UTF_8

  • IN bash, grep

    current operating installation:

    $ sudo env|grep UTF-8
LC_ALL=ru_RU.UTF-8
LANG=ru_RU.UTF-8

    Then set them to .bashrc

    correct, similar but not exactly with language ru_RU

    , for example:

    export LC_ALL=ru_RU.UTF-8
export LANG=ru_RU.UTF-8
+15


source


The .open file (yml_file, 'w') must be changed to File.open (yml_file, 'w b )



+15


source


I just endured a few hours later trying to fix a similar issue. I checked my locales, database encoding, whatever I could think of and was still getting ASCII-8BIT from the database.

Ok, it turns out that if you store text in a binary field, it will automatically be returned as ASCII-8BIT encoded text, which makes sense, however it can (obviously) cause problems in your application.

It can be fixed by changing the column encoding to :text

in your migrations.

+3


source






More articles:


All Articles