PyErr_CheckSignals not picking up signal
I have a C extension that implements the LRU cache https://github.com/pbrady/fastcache . I recently noticed that in an application (SymPy) that uses caching quite heavily, the timeout signal is lost and the application continues to run. This only happens when using my C extension, not with a clean Python LRU cache (i.e. Functools.lru_cache) https://github.com/pbrady/fastcache/issues/26 .
I overloaded my routines with PyErr_CheckSignals () calls and the signal was lost less often, but it still happens. Note that during the call, the cache will call PyObject_Hash, PyDict_Get / Set / DelItem, and PyObject_Call (in case of a miss).
Here's the relevant SymPy code snippet (timeout is an integer):
def _timeout(self, function, timeout):
def callback(x, y):
signal.alarm(0)
raise Skipped("Timeout")
signal.signal(signal.SIGALRM, callback)
signal.alarm(timeout) # Set an alarm with a given timeout
function()
signal.alarm(0) # Disable the alarm enter code here
Could something rewrite the signal? If so, how do I get around this?
source to share
It turns out there is no real mystery here, but it's a little tricky.
In addition to calls to PyErr_CheckSignals (), the signal can be caught by the interpreter whenever control passes to it through calls to PyObject_Hash, PyDict_Get / Set / DelItem. If the signal is caught by the interpreter in one of these functions, it will throw an exception due to the callback function (and the signal will disappear from the moment it is processed). However, I did not check the return value of all my functions (i.e. I knew that my argument was hashable, so I did not check the return value of PyDict_SetItem) So the exception was ignored and the program continued to execute as if the signal was not executed.
Special thanks to Ondrej for this.
source to share