How to manage GPU resources for a single worker in a distributed folder?

I have a question about dask is distributed .

Suppose I would like to run a set of tasks, each running on a different number of GPUs, for example, one task is running on two GPUs (type A), while several others are running on 1 GPU (type B). My understanding is that it is possible to configure multiple workers with the "resources" property. For example, I might have 3 GPUs in total and set up one worker with the resource variable "gpus = 2" and another with "gpus = 1". It is clear that here "gpus" is an abstract term (see here ).

However, this would mean that after a type A task is launched in a workstation with two GPUs, that worker will execute one type B job at the same time, thus not using another available GPU associated with that worker. It's right?

Ideally, I would like to define one worker with a total GPU count (here: 3) that gets updated (like ncores) when jobs run. So a worker should be able to run one type A task and one type B task at the same time, and then run three type B tasks in parallel. Is it possible?

Greetings

+3


source to share


1 answer


Yes, you can just specify that the worker has three GPUs. The worker keeps track of the resources of the currently running tasks and ensures that he does not start tasks whose total resource requirements exceed the work resources.



+2


source







All Articles