How does "harvest" work on tornado when called asynchronously?
I was researching Intro to Tornado recently and I came across the following code:
class IndexHandler(tornado.web.RequestHandler):
@tornado.web.asynchronous
@tornado.gen.engine
def get(self):
query = self.get_argument('q')
client = tornado.httpclient.AsyncHTTPClient()
response = yield tornado.gen.Task(client.fetch,
"http://search.twitter.com/search.json?" + \
urllib.urlencode({"q": query, "result_type": "recent", "rpp": 100}))
body = json.loads(response.body)
[...omitted the following code...]
Earlier I learned that yield
is a keyword that turns a generic function into a generator, and when used in the form other = yield foo
means "yield foo and when I am assigned a value, set a different value." So, I tried the following code in ipython:
In [1]: result = 'init' #set a global variable
In [2]: def test_yield():
...: global result
...: print 'start test...'
...: result = yield 'foo'
...: print 'end test...'
...:
In [3]: t = test_yield()
In [4]: t.next()
start test...
Out[4]: 'foo' #'foo' has been yield to the caller, and blocked
Now I typed the global varialbe result
and it was still referencing the 'init' line:
In [5]: print result
init
Then I called the method send()
and sent a new line to yield
:
In [6]: t.send('new message')
end test...
---------------------------------------------------------------------------
StopIteration Traceback (most recent call last)
/home/chiyu/<ipython-input-6-b86312ad7d0e> in <module>()
----> 1 t.send('new message')
StopIteration:
As expected, a StopIteration
was hoisted and printed the line "end test ...", but now the global variable has been changed result
:
In [7]: print result
new message
Explicitly, the statement yield
took a string when we called the method send()
and assigned a new string to the result to a variable.
MY QUESTION:
Let's go back to the code shown above, according to this logic,
response = yield tornado.gen.Task(client.fetch,
"http://search.twitter.com/search.json?" + \
urllib.urlencode({"q": query, "result_type": "recent", "rpp": 100}))
when the method returns client.fetch
, it will create an instance Task
and yield
for the caller, but the variable response
on the left will not get anything, because the method send()
has not been excluded. I was rather confused about this and searched in vain.
I would be very grateful for your explanations!
source to share
You already understand how Tornado uses generators to handle asynchronous calls.
I am assuming that it client
is an instance tornado.httpclient.AsyncHTTPClient()
; it fetch()
takes a callback function.
The object tornado.gen.Task
only accepts a method reference client.fetch
; you don't call it at this moment. You instantiate Task()
with this method reference and argument, and then get that.
The tornado will launch this Task
; Task
in turn will call client.fetch()
with the supplied argument plus a callback function. It then client.fetch()
executes asynchronously and invokes the callback. Anything that is then passed to the callback is recorded as the result Task
.
This result is then sent to the generator IndexHandler.get()
with the send()
returned from the expression yield Task()
and assigned response
.
In other words, Tornado uses .send()
here and is assigned something response
.
source to share
It's kind of like "I relinquish control, come back when you have the result of this future," and thus it looks like an asynchronous dereference operator. The tornado responds send
to the result when it is ready.
See http://tornado.readthedocs.org/en/latest/gen.html for a more detailed explanation .
source to share