What is the optimal number of celery workers per kernel?
There is not much information in the celery docs about the optimal number of celery workers to use on a machine. I believe that by default celery creates one worker per machine core.
I know from experimentation that starting more celery workers on one main machine is definitely beneficial (default 1 worker due to 1 core). I'm looking for a threshold at which adding more workers has a slightly diminishing return - the optimal worker number per core. I am currently using tree celery with a demon celeryd config file having the following line:
CELERYD_NODES="worker1 worker2 worker3"
My intention is to create 3 PER CORE workers (so if I started a 4 core machine there would be 12 workers). Am I doing this right or is it only starting 3 workers regardless of the number of cores?
source to share
No one has answered this question yet
Check out similar questions: