Traction / push condition in rails 3
I have longer running in the background, and how exactly can I drop the status from my background task, or would I be better off somehow passing the completion of the task to my interface?
Background:
Basically my application uses a third party service to process data, so I want this external web service work not to block all incoming requests on my website, so I put this call in a background job (I am using sidekiq). And so when this task is done, I am considering sending a webhook to a specific controller that will notify the front end of the task completion.
How can i do this? Is there a better solution for this?
Update:
My app is hosted on heroku
Update II:
I did some research on this topic and I found out that I can create a separate hero app that will handle this, found this example:
https://github.com/heroku-examples/ruby-websockets-chat-demo
This long-term task will be done for every user, on a website with a lot of traffic, is it a good idea?
source to share
I would use this with pub / sub like Faye or Pusher . The idea is that you will post the status of your long-term work on a channel, which will cause all subscribers of that channel to be notified of the status change.
For example, in your workplace, you can notify Faye of the status with something like:
client = Faye::Client.new('http://localhost:9292/')
client.publish('/jobstatus', {id: jobid, status: 'in_progress'})
And then in your frontend, you can subscribe to this feed using javascript:
var client = new Faye.Client('http://localhost:9292/');
client.subscribe('/jobstatus', function(message) {
alert('the status of job #' + message.jobid + ' changed to ' + message.status);
});
Using the pub / subsystem in this way allows you to scale your live events apart from the main application - you can run Faye on a different server. You can also take a hosted (and paid) solution like Pusher and let them take care of scaling your infrastructure.
It's also worth mentioning that Faye uses the bayeaux protocol , which means it will use websites where it is available and long polling where it isn't.
source to share
We have this model and use two different approaches. In both cases, background jobs are done with Resque , but you can probably do something similar with DelayedJob or Sidekiq .
interview
In the polling approach, we have a javascript object on the page that sets a timeout for polling with the url passed to it from the HTML rails view.
This triggers an Ajax call ("script") on the provided URL, which means Rails is looking for a JS template. So we use this to react to the state and fire an event to respond to the object when it is available or not.
It's a bit tricky and I would not recommend it at this point.
Sockets
The best solution we found was using WebSockets (with shims). In our case we are using PubNub, but there are many services for that. This prevents your web server from polling / opening a connection and is significantly more cost effective than running the servers required to handle that connection.
You stated that you are looking for frontend solutions and you can handle all frontends using the PubNub client JavaScript library.
Here's a rough idea of how we notify PubNub from the backend.
class BackgroundJob
@queue = :some_queue
def perform
// Do some action
end
def after_perform
publish some_state, client_channel
end
private
def publish some_state, client_channel
Pubnub.new(
publish_key: Settings.pubnub.publish_key,
subscribe_key: Settings.pubnub.subscribe_key,
secret_key: Settings.pubnub.secret_key
).publish(
channel: client_channel,
message: some_state.to_json,
http_sync: true
)
end
end
source to share
The simplest approach I can think of is that you set a flag in your db when the task completes, and your frontend (view) sends an ajax request periodically to check the status of the flag in the db. If the flag is set, you take the appropriate action on the view. Below are some code examples:
Since you suggested running this task for each user, so add a boolean table users
- task_complete
. When you add a job to sidekiq, you can turn off the flag:
# Sidekiq worker: app/workers/task.rb
class Task
include Sidekiq::Worker
def perform(user_id)
user = User.find(user_id)
# Long running task code here, which executes per user
user.task_complete = true
user.save!
end
end
# When adding the task to sidekiq queue
user = User.find(params[:id])
# flag would have been set to true by previous execution
# In case it is false, it means sidekiq already has a job entry. We don't need to add it again
if user.task_complete?
Task.perform_async(user.id)
user.task_complete = false
user.save!
end
In the view, you can periodically check if the flag has been set using ajax requests:
<script type="text/javascript">
var complete = false;
(function worker() {
$.ajax({
url: 'task/status/<%= @user.id %>',
success: function(data) {
// update the view based on ajax request response in case you need to
},
complete: function() {
// Schedule the next request when the current one complete, and in case the global variable 'complete' is set to true, we don't need to fire this ajax request again - task is complete.
if(!complete) {
setTimeout(worker, 5000); //in miliseconds
}
}
});
})();
</script>
# status action which returns the status of task
# GET /task/status/:id
def status
@user = User.find(params[:id])
end
# status.js.erb - add view logic based on what you want to achieve, given whether the task is complete or not
<% if @user.task_complete? %>
$('#success').show();
complete = true;
<% else %>
$('#processing').show();
<% end %>
You can set a timeout based on the average time taken to complete your task. Let them say that your task takes an average of 10 minutes, so there is no point in checking it at a frequency of 5 seconds.
Also, if the frequency of your task is something complex (not 1 per day), you can add a timestamp task_completed_at
and base your logic on a combination of flag and timestamp.
Regarding this part: "This long-term task will be done for every user, on a website with a lot of traffic, is it a good idea?"
I don't see a problem with this approach, although architectural changes such as running assignments (outsourced workers) on separate hardware will help. These are lightweight ajax calls, and some intelligence built into your javascript (like a global full flag) will avoid unnecessary requests. If you have huge traffic and DB read / write is a concern, you might want to store this flag directly in redis
instead (since you already have it for sidekiq). I believe this will solve your read / write problems and I don't see that it will cause problems. This is the simplest and cleanest approach I can think of, although you can try to achieve the same via web ports, which are supported by most modern browsers (although this might cause problems in older versions).
source to share