ZeroRPC: Continue after sending a response
I am using Python 2.7 and zeroRPC so that the client and server communicate with each other . I want the client to send a request to the server, and I want the server to send a response to confirm that it received the request. But then I want the server to do some heavy computation on this request. These calculations will take hours and will not respond, so the client does not have to wait; the client-server connection must terminate immediately after the server confirms that it has received the request. How can i do this?
Here's (oversimplification) what I have.
Server code:
impor time
import zerorpc
class HelloRPC(object):
def hey(self, name):
print 'Hey, %s' % name # just so I can check that the request was received
# send response confirming that request was received
# terminate connection
time.sleep(100000000000000) # some heavy computations
s = zerorpc.Server(HelloRPC())
s.bind('tcp://0.0.0.0:4242')
s.run()
Client code:
import zerorpc
c = zerorpc.Client()
c.connect('tcp://MyServerName:4242')
c.hey('macarena')
It doesn't work: I get zerorpc.exceptions.LostRemote: Lost remote after 10s heartbeat
. I know I can use an argument heartbeat
to make the communication last indefinitely, but as I said, the computation will take hours and give no response, so I don't think I should keep the communication online.
I've read about gevent, but I can't figure out how to use it for this purpose (is it even the right tool for the job?). Should I use the Python multiprocessing package to create a subprocess or something like that? How can you do this kind of thing?
source to share
This is because the server is not sending back a pulse. The main thread is blocked in some non-gevent-co-op cycle for about 10 seconds (2 times the default heart rate).
Gevent works alongside your coroutines (green). If the coprocessor never returns to the gentent loop, it will block the loop.
Perhaps your task is processor related, in which case you are trying to stay in co-op while fetching bits on the processor at the same time. There are two ways in which you can run your processor binding code using zerorpc (and gevent):
-
return periodically to gevent IOLoop (
gevent.sleep(0)
). Since the default heart rate is 5s, and a disconnection is expected after double heart rate, you should provide approximately every 5 seconds to be safe. -
A more general solution is to run the processor binding code in its own process (for example, one process per processor):
- One of the processes will be your web-accessible API (zerorpc.Server with heartbeat enabled). This process never blocks or takes into account joint planning.
- As many processes as you have a computationally intensive processor.
- Your service API and related processes are linked to zerorpc, but this time with the remote disabled (and probably a very long timeout value).
- In the service API process, you can control yourself with respect to the processes associated with the processor.
source to share