How does "harvest" work on tornado when called asynchronously?

I was researching Intro to Tornado recently and I came across the following code:

class IndexHandler(tornado.web.RequestHandler):
    def get(self):
        query = self.get_argument('q')
        client = tornado.httpclient.AsyncHTTPClient()
        response = yield tornado.gen.Task(client.fetch,
                "" + \
                urllib.urlencode({"q": query, "result_type": "recent", "rpp": 100}))
        body = json.loads(response.body)

        [...omitted the following code...]


Earlier I learned that yield

is a keyword that turns a generic function into a generator, and when used in the form other = yield foo

means "yield foo and when I am assigned a value, set a different value." So, I tried the following code in ipython:

In [1]: result = 'init'     #set a global variable

In [2]: def test_yield():
   ...:     global result
   ...:     print 'start test...'
   ...:     result = yield 'foo'
   ...:     print 'end test...'

In [3]: t = test_yield()

In [4]:
start test...
Out[4]: 'foo'  #'foo' has been yield to the caller, and blocked


Now I typed the global varialbe result

and it was still referencing the 'init' line:

In [5]: print result


Then I called the method send()

and sent a new line to yield


In [6]: t.send('new message')
end test...
StopIteration                             Traceback (most recent call last)
/home/chiyu/<ipython-input-6-b86312ad7d0e> in <module>()
----> 1 t.send('new message')



As expected, a StopIteration

was hoisted and printed the line "end test ...", but now the global variable has been changed result


In [7]: print result
new message


Explicitly, the statement yield

took a string when we called the method send()

and assigned a new string to the result to a variable.


Let's go back to the code shown above, according to this logic,

response = yield tornado.gen.Task(client.fetch,
                    "" + \
                    urllib.urlencode({"q": query, "result_type": "recent", "rpp": 100}))


when the method returns client.fetch

, it will create an instance Task

and yield

for the caller, but the variable response

on the left will not get anything, because the method send()

has not been excluded. I was rather confused about this and searched in vain.

I would be very grateful for your explanations!


source to share

2 answers

You already understand how Tornado uses generators to handle asynchronous calls.

I am assuming that it client

is an instance tornado.httpclient.AsyncHTTPClient()

; it fetch()

takes a callback function.

The object tornado.gen.Task

only accepts a method reference client.fetch

; you don't call it at this moment. You instantiate Task()

with this method reference and argument, and then get that.

The tornado will launch this Task

; Task

in turn will call client.fetch()

with the supplied argument plus a callback function. It then client.fetch()

executes asynchronously and invokes the callback. Anything that is then passed to the callback is recorded as the result Task


This result is then sent to the generator IndexHandler.get()

with the send()

returned from the expression yield Task()

and assigned response


In other words, Tornado uses .send()

here and is assigned something response




It's kind of like "I relinquish control, come back when you have the result of this future," and thus it looks like an asynchronous dereference operator. The tornado responds send

to the result when it is ready.

See for a more detailed explanation .



All Articles