Do I need to use args syntax in Python multiprocessing?
I am studying multiprocessing techniques in Python and came across a question. Consider the following example:
import multiprocessing as mp
def worker(n):
print('worker %d' % n)
return
if __name__ == '__main__':
jobs = []
for i in range(5):
p = mp.Process(target = worker, args = (i,))
jobs.append(p)
p.start()
This is how the following documentation uses Process
.
Does the user need args = (i,)
? I've never seen this syntax in Python before and it seems strange. I tested and this works exactly the same:
p = mp.Process(target = worker(i))
Is there some reason why I should avoid this? Thanks for any help.
source to share
Here's a quick way to prove they are not the same thing. Change the function worker(i)
like this:
import time
def worker(n):
print('worker %d' % n)
time.sleep(1)
return
When you call it the first way, you will notice that you still get all 5 prints at the same time at the beginning. When you do this in your second way, you will see that you receive all 5 prints, with the second in between.
Think about what you are trying to set up: each of the 5 independent processes starts at about the same time, each prints at about the same time, and then each one waits for about a second, but the total time passed through just over a second. This is what you want to do.
Here's the key point: target = worker
installs target
as a function worker
, and args = (i,)
installs args
as a single element containing i
. On the other hand, it target = worker(i)
calls worker(i)
and sets target
as the return value of the function, in this case None
. You don't use multiprocessing
the second way at all when you do it. If you have a really time consuming task that you want to split into multiple processes, then you won't see an improvement when the second method did it.
Basically, anytime you have func(args)
, you will call the function and get its return value, whereas when passed func
and args
individually, you allow the package multiprocessing
to run magic and make those function calls independent processes. Setting the target func(args)
will simply call the function in the main process, losing you the benefits of multiprocessing in the first place.
source to share