Is it possible to run an asyncio event loop in the background without creating a thread or process?
I have a service implemented in Python 3.5 that needs to periodically download a file over the network. I would like to avoid blocking the main thread while it is loading. To avoid introducing the complexity of threads and shared memory into my application, I wanted to use an event loop with the asyncio package to do this.
To start the event loop I found the AbstractEventLoop.run_forever()
and methods AbstractEventLoop.run_until_complete(future)
, but they both seem to block the main thread when called. The only way to avoid this is to start the loop on a different thread. But then if I'm using streams anyway, there is no point in using an event loop.
So my question is, is it possible to run an asyncio event loop in the background without creating a thread or process?
source to share
You can use the method run_in_executor
. Each of the functions that you run using this method are executed on their own thread (in parallel).
The AbstractEventLoop.run_in_executor () method can be used with a thread pool executor to execute a callback on a different thread, so as not to block the event loop thread.
The executor argument must be an Executor instance. The default artist is used if the artist is None.
For example:
import asyncio
def load_file_over_network(file_name):
# do the file loading
pass
loop = asyncio.get_event_loop()
file_name = 'FILE_NAME'
# load the file without blocking
loop.run_in_executor(None, load_file_over_network, file_name)
# do some other stuff in the main thread
# while the synchronous code above is running in other threads, the event loop
# can go do other things
# load the file again without blocking
loop.run_in_executor(None, load_file_over_network, file_name)
Hope this was helpful :)
source to share
If your entire program runs without an event asyncio
loop, it doesn't make much sense to start just one task in a new event loop. Try using a much less complex library instead concurrent.futures
. Any task invoked immediately returns an instance Future
that has methods .done()
and .result()
:
import time
from concurrent.futures.thread import ThreadPoolExecutor
def fetcher():
print("fetching...")
time.sleep(2)
print("got it!")
return 42
with ThreadPoolExecutor(max_workers=10) as executor:
fut = None
for i in range(100):
if i % 10 == 2:
fut = executor.submit(fetcher)
print(i, fut)
if fut and fut.done():
result = fut.result() # can raise an error
print("Done! Got:", result)
fut = None
time.sleep(0.5)
source to share