Celery Start a task while doing other tasks
I have 3 tasks in celery.
celery_app.send_task('tasks.read_cake_recipes')
celery_app.send_task('tasks.buy_ingredients')
celery_app.send_task('tasks.make_cake')
Both read_cake_recipes
and buy_ingredients
have no dependencies, but before the task make_cake
can be run as read_cake_recipes
, and buy_ingredients
should be completed.
make_cake
can run on ANYTIME after the first two have started. But make_cake
has no idea if other tasks have completed. Therefore, if read_cake_recipes
or buy_ingredients
takes too long, make_cake will fail.
Chains don't work here because they make_cake
have more than one dependency.
How can I start the task make_cake
, but does it then wait / wait and until the first two tasks are done?
My savings is that read_cake_recipes and buy_ingredients save the results to the DB, if make_cake somehow knows what ingredients or recipes to look for, can this check it maybe?
source to share
Fully guessing your basic architecture, but here it goes.
class Cake(models.Model):
recipes_read = models.BooleanField(default=False)
ingredients_purchased = models.BooleanField(default=False)
batter_prepared = models.BooleanField(default=False)
def save(self, *args, **kwargs):
if self.recipes_read and self.ingredients_purchased:
self.batter_prepared = True
super(Cake, self).save(*args, **kwargs)
@task
read_cake_recipes():
for cake in Cake.objects.all():
# Read some shit!
cake.recipes_read = True
cake.save()
@task
buy_cake_ingredients():
for cake in Cake.objects.all():
# Buy some shit!
cake.ingredients_purchased = True
cake.save()
@task
make_cake():
for cake in Cake.objects.filter(batter_prepared=True):
# Make that shit!
source to share