Python Multiprocessing Object Reference

I am not getting the python multiprocessing module. I start a process with a function and pass both the object as my parameter. As far as I understand, it should be an exact copy of this object. But if I try to print the address of this object, then it will be the same in every process. How is this possible? Shouldn't it be different in every process? And if this is the same object in each process, how does it happen, is it not global, but local to each process?

I have my object defined like this:

class MyClass():

    my_field = None

    def __init__():
        self.my_field = 0

      

And a function that runs in a separate process

def function_to_run_in_process(some_object):
    print some_object

      

The result for multiple processes is something like:

<__main__.MyClass instance at 0x7f276419e5f0>
<__main__.MyClass instance at 0x7f276419e5f0>
<__main__.MyClass instance at 0x7f276419e5f0>

      

etc.

If I tried to change some field of an object inside the process like this:

def function_to_run_in_process(some_object, process_lock):
   process_lock.acquire()
   some_object.some_field = some_object.some_field + 1
   process_lock.acquire()
   print some_object, 'with some_field = ', some_object.some_field, 'at address: ', hex(id(some_object.some_field))

      

I get a result similar to this:

<__main__.MyClass instance at 0x7f276419e5f0>  with some_field = 1  at address  0xc5c428>
<__main__.MyClass instance at 0x7f276419e5f0>  with some_field = 1  at address  0xc5c428>
<__main__.MyClass instance at 0x7f276419e5f0>  with some_field = 1  at address  0xc5c428>

      

So, if the passed object is just a copy, then why are there the same addresses not only for the object, but even for its field? And if they are the same, why is the field change not visible?

+3


source to share


1 answer


How is this possible?

Each process has its own virtual address space

Shouldn't the addresses be distinguished in each process?

Not. Child processes inherit the VAS of their parent. See clone and fork .

And if this is the same object in every process, how does it happen, is it not global, but local to each process?

The process memory pages in VAS will be set to COPY ON WRITE . Until the page is changed, they will all point to the same physical memory, however, if any changes are made, the VAS will map to that page elsewhere.




Here's an article on process memory management: http://duartes.org/gustavo/blog/post/anatomy-of-a-program-in-memory/


If you want to share state between processes, you can exchange memory or pass messages. The multiprocessing module can create python objects on shared memory pages, it is described here

However, it is best to avoid especially if all of the above was news to you.

+1


source







All Articles