Exception re-throwing time increases when using asyncio
I have this case where coroutines depend on another coroutine. If an exception is thrown in a given dependency, all dependents must stop and log a trace. However, as the exception re-ascends, the trace gets longer and longer.
# coding: utf-8
import asyncio
import traceback
async def resource():
return 7 / 0
async def worker(dependency):
print('waiting on a dependency')
try:
await dependency
except Exception as e:
print('Worker crashed:\n' + traceback.format_exc())
async def main(loop):
dependency = loop.create_task(resource())
workers = [loop.create_task(worker(dependency)) for _ in range(4)]
return await asyncio.wait(workers, loop=loop)
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main(loop))
loop.stop()
loop.close()
For the first run, the trace looks like this:
waiting on a dependency
Worker crashed:
Traceback (most recent call last):
File "/home/slapec/scripts/sandbox/exc.py", line 13, in worker
await dependency
File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 360, in __iter__
return self.result() # May raise too.
File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/tasks.py", line 239, in _step
result = coro.send(None)
File "/home/slapec/scripts/sandbox/exc.py", line 7, in resource
return 7 / 0
ZeroDivisionError: division by zero
After the 4th iteration, it looks like this:
waiting on a dependency
Worker crashed:
Traceback (most recent call last):
File "/home/slapec/scripts/sandbox/exc.py", line 13, in worker
await dependency
File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 360, in __iter__
return self.result() # May raise too.
File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
File "/home/slapec/scripts/sandbox/exc.py", line 13, in worker
await dependency
File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 360, in __iter__
return self.result() # May raise too.
File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
File "/home/slapec/scripts/sandbox/exc.py", line 13, in worker
await dependency
File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 360, in __iter__
return self.result() # May raise too.
File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
File "/home/slapec/scripts/sandbox/exc.py", line 13, in worker
await dependency
File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 360, in __iter__
return self.result() # May raise too.
File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/tasks.py", line 239, in _step
result = coro.send(None)
File "/home/slapec/scripts/sandbox/exc.py", line 7, in resource
return 7 / 0
ZeroDivisionError: division by zero
AFAIK this is not entirely asynchronous, but I've never seen this before.
Do you know how you can preserve the original trace? I have a feeling that stack frames are stored as well, so a long loop will consume all my memory.
+3
slapec
source
to share
No one has answered this question yet
Check out similar questions:
2020
3
2
1
0
0
0
0
0
0