The correct way to "timeout" a request in a "Tornado"
I managed to code a rather silly error that would cause one of my request handlers to execute a very slow DB query.
The interesting bit is that I noticed that even the long siege was over. Tornado was still brandishing requests (sometimes 90s later). (Comment -> I'm not 100% sure how Siege works, but I'm pretty sure it closed the connection ..)
My question is in two parts: - Is the tornado canceling the request handlers when the client closes the connection? - Is there a way to handle timeout requests in Tornado?
I have read the code and cannot find anything. Even though my request handlers are running asynchronously in the above error, the number of pending requests is reset to a level where it slows down the application and it would be better to close the connections.
source to share
Tornado does not automatically close the request handler when the client disconnects the connection. However, you can override on_connection_close
to be alerted when a client crashes, allowing you to cancel the connection at your end. A context manager (or decorator) can be used to handle setting a timeout for processing a request; use tornado.ioloop.IOLoop.add_timeout
to schedule some method that does not execute the request to run after timeout
as part of __enter__
the context manager and then cancels that callback in the __exit__
context manager block . Here's an example demonstrating both of these ideas:
import time
import contextlib
from tornado.ioloop import IOLoop
import tornado.web
from tornado import gen
@gen.coroutine
def async_sleep(timeout):
yield gen.Task(IOLoop.instance().add_timeout, time.time() + timeout)
@contextlib.contextmanager
def auto_timeout(self, timeout=2): # Seconds
handle = IOLoop.instance().add_timeout(time.time() + timeout, self.timed_out)
try:
yield handle
except Exception as e:
print("Caught %s" % e)
finally:
IOLoop.instance().remove_timeout(handle)
if not self._timed_out:
self.finish()
else:
raise Exception("Request timed out") # Don't continue on passed this point
class TimeoutableHandler(tornado.web.RequestHandler):
def initialize(self):
self._timed_out = False
def timed_out(self):
self._timed_out = True
self.write("Request timed out!\n")
self.finish() # Connection to client closes here.
# You might want to do other clean up here.
class MainHandler(TimeoutableHandler):
@gen.coroutine
def get(self):
with auto_timeout(self): # We'll timeout after 2 seconds spent in this block.
self.sleeper = async_sleep(5)
yield self.sleeper
print("writing") # get will abort before we reach here if we timed out.
self.write("hey\n")
def on_connection_close(self):
# This isn't the greatest way to cancel a future, since it will not actually
# stop the work being done asynchronously. You'll need to cancel that some
# other way. Should be pretty straightforward with a DB connection (close
# the cursor/connection, maybe?)
self.sleeper.set_exception(Exception("cancelled"))
application = tornado.web.Application([
(r"/test", MainHandler),
])
application.listen(8888)
IOLoop.instance().start()
source to share
Another solution to this problem is to use gen.with_timeout :
import time
from tornado import gen
from tornado.util import TimeoutError
class MainHandler
@gen.coroutine
def get(self):
try:
# I'm using gen.sleep here but you can use any future in this place
yield gen.with_timeout(time.time() + 2, gen.sleep(5))
self.write("This will never be reached!!")
except TimeoutError as te:
logger.warning(te.__repr__())
self.timed_out()
def timed_out(self):
self.write("Request timed out!\n")
I liked how the contextlib solution works, but I always had some logging leftovers.
A native coroutine solution would be:
async def get(self):
try:
await gen.with_timeout(time.time() + 2, gen.sleep(5))
self.write("This will never be reached!!")
except TimeoutError as te:
logger.warning(te.__repr__())
self.timed_out()
source to share