How to capture an event in Celery
I just started using celery (more specifically django-celery) and I am still not familiar with it.
I am developing an application that will dispatch tasks to be done remotely to workers, where each task takes about 3 minutes. (Not heavy processing, web crawling, but inappropriate).
When I submit an assignment, I receive a link that can be used to retrieve the results.
>>> result = task_name.delay()
>>> result
<AsyncResult: c34d78d8-b512-4165-9384-2b87933e33b7>
But in order to know when the task is finished, I need to continue polling
>>> result.ready()
False
My idea to avoid polling is to listen for the success / failure event of the task . So when I know they have returned, I can continue processing without checking the database or constantly asking if it's ready. (If anyone has a better suggestion for verifying task completion, I'd love to hear)
Not sure if I have obviousness issues, but I don't understand how to do this.
In the Celery Events docs , he says that I have to use a custom camera for this. And this is another problem as I intend to use the django-celery Django Admin Monitor.
So finally: Is it possible to record an event without a camera? How? If not, can I use it with django-celery camera?
source to share
The problem with on_success
other handlers Task
is that it works with a worker. If you use this approach, the code you add to this method should initiate the required action; but the task has no information about which process called it, so you would probably start another task ... that somehow routes to the caller who will wait in a queue to be executed ... doesn't sound what are you looking for.
To receive asynchronous task level event notifications, use Celery.events.State
as shown in http://docs.celeryproject.org/en/latest/userguide/monitoring.html?highlight=my_monitor#id20 . You can choose which events to track and be notified in real time (reverse style) when an event occurs.
source to share
Now the celery documentation is currently not resolvable for me, but it looks like you should be able to customize a callback method that your remote task can call to tell you that the task has completed. I'll update this answer if / when I can get the celery documentation.
EDIT: Get the documentation now.
So the documentation looks like what you want is a task class with a handler on_success
or after_return
. Depending on your use case, this method might do the processing you need to do using the return value of the task, or it might notify you that the result succeeded. I would recommend to the ex unless you have a specific need to run on a handler instead of a worker. Regardless, in any case, you will no longer need to interrogate result.ready()
.
source to share