Using observer pattern to remove an observer when an object has been disposed in Python
I am trying to implement a very simple observer pattern in python.
Here is my class Observer
(it's really just an interface and I think it really isn't needed):
class Observer():
def update(self,subject,message): pass
And my class Subject
(aka Observable
, but I prefer Subject
):
class Subject():
def __init__(self):
self.observers = []
def registerObserver(self, observer):
if observer not in self.observers:
self.observers.append(observer)
def removeObserver(self, observer):
self.observers.remove(observer)
def notifyObservers(self, message = None):
for observer in self.observers:
observer.update(self,message)
The class A
contains a nested class DelNotifier
that is subclassed Subject
. The idea is that when disposing of a class object A
(actually garbage collected since it is in a method __del__
) will A.DelNotifier
notify all its observers of the disposal.
class A():
def __init__(self, name):
self.name = name
self.delNotifier = A.DelNotifier(self)
class DelNotifier(Subject):
def __init__(self, outer):
super(A.DelNotifier,self).__init__()
self.outer = outer
def notifyObservers(self):
Subject.notifyObservers(self,"This is Class A object " + self.outer.name + ": I'm dying!")
def registerB(self,observer):
if not isinstance(observer,B): raise ValueError("Can only register Class B objects with Class A.")
self.delNotifier.registerObserver(observer.Aobserver)
def deleteme(self):
print("Must notify observers of my impending doom first...")
self.delNotifier.notifyObservers()
def __str__(self):
return "Class A object " + self.name
def __del__(self):
self.deleteme()
print("Done notifying everyone, time to go gentle into that good night.")
The class B
contains a nested class AObserver
that is subclassed Observer
and will receive a message from the A.DelNotifier
class object when it A
was disposed (again, this actually happens when the object A
was garbage collected):
class B():
def __init__(self, name, a):
self.name = name
self.Aobserver = B.AObserver(self)
a.registerB(self)
class AObserver(Observer):
def __init__(self,outer):
super(B.AObserver,self).__init__()
self.outer = outer
def update(self,subject,message):
print(str(self.outer) + " received message: '" + str(message) + "'")
print("Time for", self.outer, "to die, too.")
self.outer.__del__()
def __str__(self):
return "Class B object " + self.name
def __del__(self):
print("This is " + str(self) + ": now I'm dying, too!")
This construct works when I call __del__()
directly, however some of the objects seem to be gc'd the second time the session ends:
>>> a = A('a')
>>> b1 = B('b1', a)
>>> b2 = B('b2', a)
>>> a.__del__()
Must notify observers of my impending doom first...
Class B object b1 received message: 'This is Class A object a: I'm dying!'
Time for Class B object b1 to die, too.
This is Class B object b1: now I'm dying, too!
Class B object b2 received message: 'This is Class A object a: I'm dying!'
Time for Class B object b2 to die, too.
This is Class B object b2: now I'm dying, too!
Done notifying everyone, time to go gentle into that good night.
>>> exit()
Must notify observers of my impending doom first...
Class B object b1 received message: 'This is Class A object a: I'm dying!'
Time for Class B object b1 to die, too.
This is Class B object b1: now I'm dying, too!
Class B object b2 received message: 'This is Class A object a: I'm dying!'
Time for Class B object b2 to die, too.
This is Class B object b2: now I'm dying, too!
Done notifying everyone, time to go gentle into that good night.
This is Class B object b1: now I'm dying, too!
This is Class B object b2: now I'm dying, too!
Another problem, and I think this is more important, is that when I have del
an element of a class A
from a list, the element is not immediately garbage collected, and I cannot be sure that any registered elements B
have been removed:
>>> b1 = B('b1',a[0])
>>> b2 = B('b2',a[0])
>>> del a[0]
## Note that items are not deleted until session exits
>>> exit()
Must notify observers of my impending doom first...
Class B object b1 received message: 'This is Class A object a: I'm dying!'
Time for Class B object b1 to die, too.
This is Class B object b1: now I'm dying, too!
Class B object b2 received message: 'This is Class A object a: I'm dying!'
Time for Class B object b2 to die, too.
This is Class B object b2: now I'm dying, too!
Done notifying everyone, time to go gentle into that good night.
##Note that the class B objects get gc'd a second time....???
This is Class B object b1: now I'm dying, too!
This is Class B object b2: now I'm dying, too!
In addition to these problems, I am aware of many problems with using a method __del__
to do anything other than clearing an object after it has been gc'd, and that it should probably be avoided for the purposes I am trying to use ... But I don't know any other way.
What would be the best way to do this? I was thinking about trying to use the context manager ( with
) to remove things after I finished using them, but I have no experience with that. If this were a good option, how would I go about it? What would it look like?
EDIT: Clarifying the desired behavior
I'll try to clear up some (understandable) confusion.
I simplified the code a bit, but B
- it is an object that depends on an object A
. If a B
A
leaves, then it B
should disappear. I will have a container (using list
here) As and Bs:
As = [A('a'+str(i)) for i in range(10)]
Bs = [B('b'+str(i),As[i]) for i in range(10)] #Bs made of As
del As[0] #whoops, don't need As[0] anymore
assert Bs[0] is None #ERROR!
#or using pop:
As.pop(0)
assert Bs[0] is None #ERROR!
Also see my previous question from last day that helped me with this whole idea to use the observer pattern in the first place.
source to share
This is a significant difference in code size to accommodate your requirement to automatically maintain a link list and clean up when links are removed. I added a class Manager
to accomplish this, and a second event deleted()
, which is a hook for the manager to clean up the lists it supports. I'm rewriting the complete, modified code here as it wasn't trivial to update my previous answer.
I believe this completely satisfies the question you asked. Perhaps not the reason you need it in the first place, but I think you asked this in another question.
We need weak links to make this work:
import weakref
The interface Observer
gets a new method that is called afterdeleted()
class Observer(object):
def update(self,subject,message): pass
def deleting(self,subject):
''' the subject is being deleted '''
pass
def deleted(self,subject):
pass
The manager's class stores lists of items and observers
class Manager(Observer):
def __init__(self, ):
self._subjects = []
self._observers = []
def monitorSubject(self, subject):
self._subjects.append( weakref.ref(subject) )
# observe the subject for deletion to
# trigger list maintenance on "deleted"
subject.registerObserver(self)
def monitorObserver(self, observer):
self._observers.append( weakref.ref(observer) )
def deleted(self, subject):
''' a subject was deleted, remove it from the list.
deleting() is called first, and the observers delete themselves.
deleted() is called next, and is a hook for the manager to
cleanup any dead subjects and observers '''
# calling a weakref returns the original object, and `None` when the
# reference is dead
def isNotDead(r):
return not r()==None
# remove any dead subjects
print 'Removing dead subjects...'
self._subjects = filter(isNotDead, self._subjects)
# remove any dead observers
print 'Removing dead observers...'
self._observers = filter(isNotDead, self._observers, )
def __str__(self,):
return "[Manager] Subjects:{0}, Observers:{1}".format(
','.join([str(r()) for r in self._subjects]),
','.join([str(r()) for r in self._observers])
)
The differences in topic are noted in the comments, but above all there is a second pass to invoke deleted
. The first pass deleting
notifies the observers, and the second pass deleted
notifies the manager. In addition, the routine __del__
uses weak references to iterate over, as some of them are removed along the way.
class Subject(object):
def __init__(self):
self.observers = []
def __del__(self, ):
''' on deletion, notify the observers '''
print "{0}.__del__".format(self)
# copy the observer list, then remove all references to the observers
# NEW - use weakrefs here, or they will not be properly deleted later
obs = [weakref.ref(o) for o in self.observers]
# clear all strong references to the observers
self.observers = []
# notify all observers that we were deleted
# ** only if they are not already deleted **
for o in obs:
if not o() == None:
o().deleting(self)
# NEW - second pass to allow the Manager to cleanup
# ** only if they are not already deleted **
for o in obs:
if not o() == None:
o().deleted(self)
def registerObserver(self, observer):
if observer not in self.observers:
self.observers.append(observer)
def removeObserver(self, observer):
self.observers.remove(observer)
def notifyObservers(self, message = None):
for observer in self.observers:
observer.update(self,message)
Same as before, with simplified string formatting
class A(Subject):
''' A is just a subject '''
def __init__(self, name):
super(A,self).__init__()
self.name = name
def __str__(self):
return "A[ {0} ]".format(self.name)
B is the same as before
class B(Observer):
''' class B is an observer of A '''
def __init__(self, name, a):
self.name = name
# don't keep a reference to 'a', or 'a' will not be deleted!
a.registerObserver(self)
def __str__(self):
return "B[ {0} ]".format(self.name)
def __del__(self):
print("{0}.__del__".format(self))
def deleting(self, subject):
''' notification from the subject (A) that it is being deleted. I'm
assuming here that the subject is actually the one we registered in
__init__, since I couldn't store a reference or else __del__ would
not have been called! '''
print "B[{0}].deleting, subject={1}".format(self.name, subject)
del self
Some code to execute the file:
if __name__ == '__main__':
mgr = Manager()
# keep strong references to the subjects, because
# we will delete them explicitly
a1 = A('a1')
a2 = A('a2')
mgr.monitorSubject(a1)
mgr.monitorSubject(a2)
# monitor observers directly, and do NOT keep
# strong references or they will not be deleted
mgr.monitorObserver( B('b1', a1) )
mgr.monitorObserver( B('b2', a1) )
mgr.monitorObserver( B('b3', a2) )
mgr.monitorObserver( B('b4', a2) )
# show the starting state
print mgr
print "Deleting a1..."
del a1
print mgr
print "Deleting a2..."
del a2
print mgr
And the output is:
# OUTPUT (some newlines added)
#
# [Manager] Subjects:A[ a1 ],A[ a2 ], Observers:B[ b1 ],B[ b2 ],B[ b3 ],B[ b4 ]
#
# Deleting a1...
# A[ a1 ].__del__
# B[ b1 ].__del__
# B[ b2 ].__del__
# Removing dead subjects...
# Removing dead observers...
# [Manager] Subjects:A[ a2 ], Observers:B[ b3 ],B[ b4 ]
#
# Deleting a2...
# A[ a2 ].__del__
# B[ b3 ].__del__
# B[ b4 ].__del__
# Removing dead subjects...
# Removing dead observers...
#
# [Manager] Subjects:, Observers:
source to share
Caveat: I work for 2.7.3
, so some might be slightly different.
I have some code that seems to work a little better than yours. Hopefully this brings up some important points.
First, the observer interface shows an event deleting
that will be triggered when the object leaves. As you noted, this is really not required, but I like to declare my interfaces explicitly.
class Observer(object):
def update(self,subject,message): pass
def deleting(self,subject):
''' the subject is being deleted '''
pass
The Subject class will then be responsible for processing __del__
and notifying all observers. This is a little oversimplified and takes the code away from A
and gets rid of all the inner classes from your original example. In the method, Subject
__del__
I copy the observers, clear them (remove all references), then call the method deleting
to let the observers remove themselves.
class Subject(object):
def __init__(self):
self.observers = []
def __del__(self, ):
''' on deletion, notify the observers '''
print "Subject.__del__".format(self)
# copy the observer list, then remove all references to the observers
obs = self.observers[:]
self.observers = []
# notify all observers that we were deleted
for o in obs:
o.deleting(self)
def registerObserver(self, observer):
if observer not in self.observers:
self.observers.append(observer)
I forgot removeObserver
and notifyObservers
because I am not using them in this solution (obviously they are useful otherwise).
The class A
is just simple Subject
and it doesn't provide anything other than a naming function. I removed the limitation I B
can only register for A
for simplicity, and I don't think this is necessary to fix the problem.
class A(Subject):
''' A is just a subject '''
def __init__(self, name):
super(A,self).__init__()
self.name = name
def __str__(self):
return "A[name={0}]".format(self.name)
Then B
, since the observer automatically adds himself as observer A when created. And when the watched entity is removed, the notification is received using the method deleting
. Since there is only one observable, I assume this is A
passed in __init__
and removed self
. It is important (at least in version 2.7) not to store references to A
inside B
, or it seems that __del__
gc can never be called (this may be my misunderstanding gc
).
class B(Observer):
''' class B is an observer of A '''
def __init__(self, name, a):
self.name = name
# don't keep a reference to 'a', or 'a' will not be deleted!
a.registerObserver(self)
def __str__(self):
return "B[name={0}]".format(self.name)
def __del__(self):
print("{0}.__del__".format(self))
def deleting(self, subject):
''' notification from the subject (A) that it is being deleted. I'm
assuming here that the subject is actually the one we registered in
__init__, since I couldn't store a reference or else __del__ would
not have been called! '''
print "B.deleting, subject={0}".format(subject)
del self
With this code, I get the output:
>>> from main import A, B
>>> a = A('a')
>>> B('b1', a)
<main.B object at 0x00000000022938D0>
>>> B('b2', a)
<main.B object at 0x0000000002293B70>
>>> del a
Subject.__del__
B.deleting, subject=A[name=a]
B.deleting, subject=A[name=a]
B[name=b1].__del__
>>> import gc
>>> gc.collect()
B[name=b2].__del__
0
which is more like what you want, except it b2
doesn't collect garbage right away.
EDIT: Added example based on further clarification of behavior
>>> from main import A, B
>>>
>>> As = [A('a'+str(i)) for i in range(10)]
>>> Bs = [B('b'+str(i),As[i]) for i in range(10)]
Removing an item from an array As
removes the item from the list and thus decreases the refcount and the object a1
is removed, which notifies b1
that the object has been removed. b1
then calls del
by itself, but not deleted due to a reference in the array Bs
.
>>> del As[1]
Subject.__del__
B.deleting, subject=A[name=a1]
Setting a reference in an array Bs
to None
zero will decrease the reference count to 0 and thus cause__del__
>>> Bs[1] = None
B[name=b1].__del__
Showing that the array Bs
needs some cleanup, manually
>>> len(As)
9
>>> len(Bs)
10
>>> del Bs[1]
>>> len(Bs)
9
And what gc
has no work:
>>> import gc
>>> gc.collect()
0
The rest are cleared by exit()
source to share