Setattr, object deletion and cyclic garbage collection
I would like to understand how deleting an object works in python. Here's a very simple bunch of code.
class A(object):
def __init__(self):
setattr(self, "test", self._test)
def _test(self):
print "Hello, World!"
def __del__(self):
print "I'm dying!"
class B(object):
def test(self):
print "Hello, World!"
def __del__(self):
print "I'm dying"
print "----------Test on A"
A().test()
print "----------Test on B"
B().test()
Pythonista will know that I am running python 2.x version. More specifically, this code works on a python 2.7.1 installation.
This code outputs the following:
----------Test on A
Hello, World!
----------Test on B
Hello, World!
I'm dying
Surprisingly, the object is A
not removed. I can understand why, since the operator setattr
in __init__
creates a circular reference. But this one seems to be easy to resolve.
Finally, this page, in the python documentation (circular garbage collection support) , shows that one can deal with this type of circular reference.
I'd like to know:
- Why do I never go through my method
__del__
in the classA
? - If my diagnosis about circular reference is good, why
object
doesn't my subclass support circular garbage collection? - finally how to deal with this type
setattr
if i really want to get through__del__
?
Note. Q A
, if setattr
points to another method of my module, no problem.
source to share
Fact 1
Instance methods are usually stored in a class. The interpreter first looks at them in the instance __dict__
that fails and then looks at the class that succeeds.
When you dynamically set an instance method A
to __init__
, you create a reference to it in the instance dictionary. This reference is circular, so the refcount will never go to zero and the check counter will not clear A
up.
>>> class A(object):
... def _test(self): pass
... def __init__(self):
... self.test = self._test
...
>>> a = A()
>>> a.__dict__['test'].im_self
Fact 2
The garbage collector is what Python uses to deal with circular references. Unfortunately, it cannot process objects with methods __del__
, since in general it cannot define a safe order to call them. Instead, it just puts all such objects in gc.garbage
. Then you can look there to break the loops so that they can be freed. From the docs
gc.garbage
A list of objects that the collector considers unavailable, but may not be freed (unclaimed objects). By default, this list contains only objects with methods
__del__()
. Objects that have methods__del__()
and are part of a reference loop cause the entire reference loop to be hopeless, including objects not necessarily in the loop, but reachable only from it. Python does not collect these automatically because, in general, it is impossible for Python to guess the safe order of running methods.__del__()
... if you know the safe ordering, you can force the problem by examining the garbage list and the explicit loop splitting due to your objects within the list. Note that these objects are kept alive even by being in the garbage list, so they should also be removed from the garbage. For example, after interrupting the loop, dodel gc.garbage[:]
to delete the list. In general, it is best to avoid the problem by not creating loops containing objects with methods__del__()
, andgarbage
you can investigate in this case to verify that such loops are not created.
therefore
Don't make circular references to objects with methods __del__
if you want them to be garbage collected.
source to share
You should carefully read the documentation for the method__del__
- in particular, the part where objects with methods __del__
change the way the collector works.
The gc module provides some hooks where you can clean them up yourself.
I suspect that simply not having a method __del__
will cause your object to be properly cleaned up. You can check this by looking gc.garbage
and seeing if your instance is present A
.
source to share