Exception re-throwing time increases when using asyncio

I have this case where coroutines depend on another coroutine. If an exception is thrown in a given dependency, all dependents must stop and log a trace. However, as the exception re-ascends, the trace gets longer and longer.

# coding: utf-8

import asyncio
import traceback

async def resource():
    return 7 / 0

async def worker(dependency):
    print('waiting on a dependency')

    try:
        await dependency
    except Exception as e:
        print('Worker crashed:\n' + traceback.format_exc())


async def main(loop):
    dependency = loop.create_task(resource())

    workers = [loop.create_task(worker(dependency)) for _ in range(4)]

    return await asyncio.wait(workers, loop=loop)


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main(loop))

    loop.stop()
    loop.close()

      

For the first run, the trace looks like this:

waiting on a dependency
Worker crashed:
Traceback (most recent call last):
  File "/home/slapec/scripts/sandbox/exc.py", line 13, in worker
    await dependency
  File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 360, in __iter__
    return self.result()  # May raise too.
  File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "/home/slapec/scripts/sandbox/exc.py", line 7, in resource
    return 7 / 0
ZeroDivisionError: division by zero

      

After the 4th iteration, it looks like this:

    waiting on a dependency
Worker crashed:
Traceback (most recent call last):
  File "/home/slapec/scripts/sandbox/exc.py", line 13, in worker
    await dependency
  File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 360, in __iter__
    return self.result()  # May raise too.
  File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/home/slapec/scripts/sandbox/exc.py", line 13, in worker
    await dependency
  File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 360, in __iter__
    return self.result()  # May raise too.
  File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/home/slapec/scripts/sandbox/exc.py", line 13, in worker
    await dependency
  File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 360, in __iter__
    return self.result()  # May raise too.
  File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/home/slapec/scripts/sandbox/exc.py", line 13, in worker
    await dependency
  File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 360, in __iter__
    return self.result()  # May raise too.
  File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/home/slapec/.pyenv/versions/3.5.1/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "/home/slapec/scripts/sandbox/exc.py", line 7, in resource
    return 7 / 0
ZeroDivisionError: division by zero

      

AFAIK this is not entirely asynchronous, but I've never seen this before.

Do you know how you can preserve the original trace? I have a feeling that stack frames are stored as well, so a long loop will consume all my memory.

+3


source to share





All Articles