Local variable is not updated in a loop in the same way as shared memory objects in Python
In the following Python code, the multiprocessing module starts three processes that print the values of one local variable and two shared memory objects with multiple processes.
import multiprocessing as mp
import os,time
# local variable
count = 0
# shared memory objects (int and array)
scalar = mp.Value('i', 0)
vector = mp.Array('d', 3)
def showdata(label, val, arr):
print(label, "==> PID:", os.getpid(), ", count:", count, ", int:", val.value, ", arr:", list(arr))
ps = []
for i in range(3):
count += 1
scalar.value += 1
vector[i] += 1
p=mp.Process(target=showdata, args=(('process %s' % i), scalar, vector))
p.start()
ps.append(p)
# time.sleep(.1)
# block the main thread until all processes have finished...
for p in ps:
p.join()
The output for this code is as follows ...
process 0 ==> PID: 35499 , count: 1 , int: 3 , arr: [1.0, 1.0, 1.0]
process 1 ==> PID: 35500 , count: 2 , int: 3 , arr: [1.0, 1.0, 1.0]
process 2 ==> PID: 35501 , count: 3 , int: 3 , arr: [1.0, 1.0, 1.0]
If I change the code to add a delay by uncommenting the time.sleep (0.1) object, then the output changes to the following:
process 0 ==> PID: 35499 , count: 1 , int: 1 , arr: [1.0, 0.0, 0.0]
process 1 ==> PID: 35500 , count: 2 , int: 2 , arr: [1.0, 1.0, 0.0]
process 2 ==> PID: 35501 , count: 3 , int: 3 , arr: [1.0, 1.0, 1.0]
It makes sense that without any delay (i.e. the first output above) the shared memory objects will have the same values for all three processes, since once they are started, the "for" loop completes quickly and updates the shared objects, the values before the individual processes can execute its target functions "showdata".
However, I don't see why the local variable "count" is allowed to be updated in stages. I expected this to be treated as shared memory objects where, without any delay, the count would increase three times faster than the "showdata" functions would run in separate processes. By this logic, "count" should be 3 for all three processes.
Can anyone explain why this is not happening?
This works in Python 3.4.3 on OS X 10.10.3.
source to share
I believe this is because it mp.Process
starts a new process (note that this is not a new thread of the same process, its a completely new process with its own PID, as you can see in your output) for each of the function, with each process having its own own memory (stack / heap). Local variables are stored in each process in their own memory, so when this particular process is called, it accesses its own stack that contains count
that was present when the process was started.
But for shared memory created by a multiprocessing thread, they are shared between each of the child processes that were spawned multiprocessing.Process
and the parent process.
source to share
The different behavior that you see from your code is due to the fact that it has a race condition. The race is between the main process that updates the shared memory values and each subprocess that prints those values. Depending on the timings of the different parts of the code, you can get any of several different results (the two you show are edge cases and are arguably the easiest to get, but this is not impossible for some, but not all, sub-processes. general data, while it is only partially updated). Try adding some random delay to the function showdata
and you may end up with a few more options.
The reason a "local" variable (which is actually a global variable, but which is not very important) does not behave the same is because it is copied into each subprocess memory as part of the call mp.Process
. There is no race condition, as it is not possible for the parent process to run ahead and change the value again before the sub process gets its copy.
source to share