Resque with Rails 3 + Heroku + RedisToGo gives Postgresql message
I am writing an application that should send a lot of emails and generate many user notifications due to these emails. This task creates a timeout in Heroku. To solve this problem, I decided to use Resque and RedistToGo.
What I did was send an email (this is actually just one email, because we are using Sendgrid to process it) and generate notifications using the Resque work resource. The email has already been created, so I am sending it to the worker ID along with all recipients.
This works fine locally. In production, unless we restart our application in Heroku, it only runs once. I'll post some of my code and error message:
#lib/tasks/resque.rake
require 'resque/tasks'
task "resque:setup" => :environment do
ENV['QUEUE'] = '*'
end
desc "Alias for resque:work (To run workers on Heroku)"
task "jobs:work" => "resque:work"
#config/initalizers/resque.rb
ENV["REDISTOGO_URL"] ||= "redis://redistogo:some_hash@some_url:some_number/"
uri = URI.parse(ENV["REDISTOGO_URL"])
Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
Dir["#{Rails.root}/app/workers/*.rb"].each { |file| require file }
#app/workers/massive_email_sender.rb
class MassiveEmailSender
@queue = :massive_email_queue
def self.perform(email_id, recipients)
email = Email.find(email_id.to_i)
email.recipients = recipients
email.send_email
end
end
I have an email model that has an after_create that enqueues a worker:
class Email < ActiveRecord::Base
...
after_create :enqueue_email
def enqueue_email
Resque.enqueue(MassiveEmailSender, self.id, self.recipients)
end
...
end
This email model also has a method send_email
that does what I said earlier
I am getting the following error. I am going to post all the information Resque gives me:
Worker
9dddd06a-2158-464a-b3d9-b2d16380afcf:1 on massive_email_queue at just now
Retry or Remove
Class
MassiveEmailSender
Arguments
21
["some_email_1@gmail.com", "some_email_2@gmail.com"]
Exception
ActiveRecord::StatementInvalid
Error
PG::Error: SSL error: decryption failed or bad record mac : SELECT a.attname, format_type(a.atttypid, a.atttypmod), d.adsrc, a.attnotnull FROM pg_attribute a LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum WHERE a.attrelid = '"emails"'::regclass AND a.attnum > 0 AND NOT a.attisdropped ORDER BY a.attnum
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:1139:in `async_exec'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:1139:in `exec_no_cache'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:663:in `block in exec_query'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/abstract_adapter.rb:280:in `block in log'
/app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.2/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/abstract_adapter.rb:275:in `log'
/app/vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.3.2/lib/new_relic/agent/instrumentation/active_record.rb:31:in `block in log_with_newrelic_instrumentation'
/app/vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.3.2/lib/new_relic/agent/method_tracer.rb:242:in `trace_execution_scoped'
/app/vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.3.2/lib/new_relic/agent/instrumentation/active_record.rb:28:in `log_with_newrelic_instrumentation'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:662:in `exec_query'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:1264:in `column_definitions'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/postgresql_adapter.rb:858:in `columns'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/connection_adapters/schema_cache.rb:12:in `block in initialize'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/model_schema.rb:228:in `yield'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/model_schema.rb:228:in `default'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/model_schema.rb:228:in `columns'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/model_schema.rb:237:in `columns_hash'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/relation/delegation.rb:7:in `columns_hash'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/relation/finder_methods.rb:330:in `find_one'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/relation/finder_methods.rb:311:in `find_with_ids'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/relation/finder_methods.rb:107:in `find'
/app/vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.2/lib/active_record/querying.rb:5:in `find'
/app/app/workers/massive_email_sender.rb:5:in `perform'
According to this, the first argument is the email id and the second is a list of all recipients ... exactly as it should be.
Can anyone help me? Thank!
source to share
I faced the same problem. Assuming you are using Active Record, you should call ActiveRecord::Base.establish_connection
Resque for each forked worker to make sure it does not have a stale database connection. Try to put this in yourlib/tasks/resque.rake
task "resque:setup" => :environment do
ENV['QUEUE'] = '*'
Resque.after_fork = Proc.new { ActiveRecord::Base.establish_connection }
end
source to share