Multiprocessing: long and short workers
I am trying to do something multi-processor with a distributed queue. Is there a reason to choose run
over run2
? My feeling is a run2
little more effective as it doesn't have the overhead of returning control to the main process.
from multiprocessing import Pool
from foo import my_distributed_queue as queue
def process_msg(msg):
foo(msg)
def run():
pool = Pool(4)
for msg in queue:
pool.apply_async(process_msg, (msg, ))
def process_msg2():
for msg in queue:
foo(msg)
def run2():
pool = Pool(4)
for i in range(4):
pool.apply_async(process_msg2, ())
if __name__ == '__main__'
run()
#run2()
+3
source to share
No one has answered this question yet
Check out similar questions: