Catching objects causing multiple SQLite access errors
I got this familiar sight while running my unit tests in Python and SQLite:
SQLite objects created in a thread can only be used in that same thread.
Tests run fine, but errors are still printed. I am guessing that the SQLite objects are somehow leaking into the background thread.
Instead of analyzing a piece of code piece by piece, there is a simple way
-
Place
pdb
a breakpoint somewhere and see which thread, which object is causing the violation (and thus immediately figure out how it ended up there). -
Print out the objects (and the parents that refer to them) somehow with an error message
source to share
Until now, looking at the pysqlite source, I can say that it is impossible to exchange references or cursor objects (see pysqlite_check_thread
function usage).
pysqlite_check_thread
the function raises an ProgrammingError
exception with this message
SQLite objects created in a thread can only be used in that same thread.
Some functions in your source code catch this exception and print it.
To find places in the source code that call connection methods in other threads, I would suggest writing a call wrapper over the connection object something like this:
# script name: sqllitethread.py
import inspect
import sqlite3
import threading
import thread
from sqlite3 import ProgrammingError
class CallWrapper(object):
def __init__(self, obj):
self.obj = obj
self.main_thread_id = thread.get_ident()
def __getattr__(self, name):
if self.main_thread_id != thread.get_ident():
print "Thread %s requested `%s` attribute from %s" % (thread.get_ident(), name, self.obj)
for frame in inspect.getouterframes(inspect.currentframe())[1:]:
if frame[1].endswith('threading.py'):
continue
print "\t", "%s:%s" % (frame[1], frame[2]), frame[3], frame[4][0].strip()
return getattr(self.obj, name)
conn = CallWrapper(sqlite3.connect('example.db'))
c = conn.cursor()
def worker():
try:
conn.execute('.tables')
except ProgrammingError, e:
print e
t = threading.Thread(target=worker)
t.start()
t.join()
Output example:
Thread 140390877370112 requested `execute` attribute from <sqlite3.Connection object at 0x7faf4e659858>
sqllitethread.py:30 worker conn.execute('.tables')
SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 140390912665408 and this is thread id 140390877370112
source to share