Clever Geek Handbook

Ruby on Rails - UTF8 encoding issues in MySQL from ActiveRecord

I have a webapp using Ruby 1.9 and Rails 4. My local VM (ubuntu) is fine. My DB and tables use utf8_unicode_ci and the data is stored well in tables and printed well on webapp pages. My problem is with my production server (EB on AWS). I am using MySQL and my database was in Swedish format, so I converted it to UTF8 with the following command:

# for my database
ALTER DATABASE xxx CHARACTER SET utf8 COLLATE utf8_unicode_ci; 

# for each tables in the DB (didn’t affect a join table that I have but doesn’t matter)
ALTER TABLE xxx CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;

But all data containing special characters from my rails shape is saved with question marks like caract? res sp? ciaux . But when I display the text on the page, the special characters are displayed correctly.

Also, I imported the list of languages ​​into a table, so the languages ​​are stored correctly in the DB (e.g. français, tchèque), but when I fetch these values ​​from web pages I got: franÃsais , tchèque

I tried 4 solutions I found online (maybe not in this order):

  • I added config.encoding = "utf-8" to config / application.rb
  • I added encoding: utf8 to config / database.rb
  • I added these two lines before initialization!

config / environment.rb

Encoding::default_internal = Encoding::UTF_8
Encoding::default_external = Encoding::UTF_8`
  1. I added a filter to my application controller

application / controllers / application_controller.rb

  before_filter :configure_charsets
  # Prevent CSRF attacks by raising an exception.
  # For APIs, you may want to use :null_session instead.
  protect_from_forgery with: :exception
  def configure_charsets
    response.headers["Content-Type"] = "text/html; charset=utf-8"
    suppress(ActiveRecord::StatementInvalid) do 
      ActiveRecord::Base.connection.execute 'SET NAMES UTF8'
    end
  end

Now I have all this configuration, but my problem remains the same. It looks like these changes had no impact.

+3


source to share


4 answers


The problem was with my terminal. I imported a list of languages ​​through it and I think this is what caused my encoding problem.



I downloaded Sequel Pro to manage my database and all my data is intact.

0


source


Try to create a database with utf parameters:



CREATE DATABASE `mydb` CHARACTER SET utf8 COLLATE utf8_general_ci;
+1


source


The connection between RoR and MySQL must be in utf8. Do the equivalent of RoR SET NAMES utf8

.

+1


source


I've had similar problems in the past and fixed with this command:

mysqldump -u root -p -h localhost <your db> --opt --skip-set-charset --default-character-set=latin1 /path/to/dump.sql

And then import the dump into your existing database.

Seems counter-intuitive, but it works for me. Make sure you have a backup (I'm sure you've taken care of this already, but I feel bad about leaving a disclaimer).

0


source






More articles:


All Articles