Docker + rails + redis - rescuers not working
I have created a docker environment that creates 3 images: rails, postgresql and redis. It works well enough, but I found my redis image had no workers.
Docker info
My docker-compose.yml looks like this
web:
build: .
command: bundle exec unicorn -p 3000 -c config/unicorn.rb
volumes:
- .:/fitmo
- ../fitmo-core:/fitmo-core
ports:
- "3000:3000"
links:
- db
- redis
environment:
- REDIS_URL=redis://redis:6379
db:
build: ./db/docker-files
ports:
- "5432"
redis:
image: redis:2.8
ports:
- "6379"
Resque config
require 'resque'
require 'resque-scheduler'
require 'resque_scheduler/server'
require 'appsignal/integrations/resque'
require 'yaml'
if Rails.env.test?
require 'mock_redis'
$redis = MockRedis.new
else
uri = URI.parse(ENV['REDIS_URL'])
$redis = Redis.new(host: uri.host, port: uri.port, password: uri.password)
end
Resque.redis = $redis
Resque.schedule = YAML.load_file(File.join(Rails.root, 'config/resque_schedule.yml'))
Resque.before_fork do
defined?(ActiveRecord::Base) and
ActiveRecord::Base.connection.disconnect!
end
Resque.after_fork do
defined?(ActiveRecord::Base) and
ActiveRecord::Base.establish_connection
end
module Resque
def queued_jobs
queues = Hash.new
Resque.queues.each do |queue|
jobs = []
Resque.peek(queue, 0, 5000).each do |job|
jobs.push({klass: job['class'], args: job['args']})
end
queues[queue] = jobs
end
queues
end
def queued?(klass, *args)
queued_jobs.select do |job|
job[:klass] == klass.to_s && job[:args] == args
end.any?
end
def enqueue_once(klass, *args)
enqueue(klass, *args) unless queued?(klass, *args)
end
end
Ruby Gems
Below are the matching gems. The commented versions are the newest versions of the gems, but I don't think this is a problem as the current versions work fine in Heroku environments. I also have a rescue mail and a planner for thoroughness. They don't play for the question
gem 'redis', '3.0.7' #'~> 3.2.0'
gem 'resque', '~> 1.25.2' #'~> 1.25.2'
gem 'resque_mailer', '~> 1.0.1' #'~> 2.2.7'
gem 'resque-scheduler', '~> 2.5.5' #'~> 4.0.0'
Testing information
I have confirmed that redis is available from the web container by telnetting to the redis host on port 6379. I also output the Rescue.info value which shows there are items in the queue, but now the workers are running
resque info: {:pending=>1, :processed=>0, :queues=>2, :workers=>0, :working=>0, :failed=>0, :servers=>["redis://redis:6379/0"], :environment=>"development"}
I am stuck at this point as I am not sure how to get the workers to work. Any suggestions?
source to share
I was able to solve this problem by including a new image to launch based on the web image. My new docker-compose.yml looks like this (new working image added):
web:
build: .
command: bundle exec unicorn -p 3000 -c config/unicorn.rb
volumes:
- .:/fitmo
- ../fitmo-core:/fitmo-core
ports:
- "3000:3000"
links:
- db
- redis
environment:
- REDIS_URL=redis://redis:6379
worker:
build: .
command: bundle exec rake environment resque:work QUEUE=*
volumes:
- .:/fitmo
- ../fitmo-core:/fitmo-core
links:
- db
- redis
environment:
- REDIS_URL=redis://redis:6379
db:
build: ./db/docker-files
ports:
- "5432"
redis:
image: redis:latest
ports:
- "6379"
If you have the AppSignal gem installed and are extending your workspaces with the "Appsignal :: Integrations :: ResquePlugin" extension, the problem is that the ResquePlugin is trying to write the socket to the tmp folder, and Docker doesn't allow it.
I'm sure there is a Docker config to allow socket writing, but instead I created a development section in appsignal.yml and set the active flag to false. Latest version gem "Connect" still tries to write to the socket even if active = false. Version 0.13.0 of the gem (to be released tomorrow) should have a fix for this in place
appsignal.yml
production:
api_key: "xxxxxxxxxxxxxxxxxxxxxxxx"
active: true
slow_request_threshold: 200
staging:
api_key: "xxxxxxxxxxxxxxxxxxxxxxxx"
active: true
slow_request_threshold: 200
development:
api_key: "xxxxxxxxxxxxxxxxxxxxxxxx"
active: false
slow_request_threshold: 200
source to share
On the resque github page https://github.com/resque/resque he mentions that you need to run bin/resque work
to start polling the queue and novice workers.
source to share