Deploying aiohttp.web app using gunicorn and nginx
I am trying to deploy aiohttp web application but cannot figure out how to get the application to work over a unix socket, which I think I need to get nginx and gunicorn to talk to each other.
A simple example application from the aiohttp documentation saved as app.py:
import asyncio
from aiohttp import web
@asyncio.coroutine
def hello(request):
return web.Response(body=b'Hello')
app = web.Application()
app.router.add_route('GET', '/', hello)
if __name__ == "__main__":
loop = asyncio.get_event_loop()
handler = app.make_handler()
f = loop.create_server(handler, '0.0.0.0', 8080)
srv = loop.run_until_complete(f)
try:
loop.run_forever()
except KeyboardInterrupt:
pass
finally:
loop.run_until_complete(handler.finish_connections(1.0))
srv.close()
loop.run_until_complete(srv.wait_closed())
loop.run_until_complete(app.finish())
loop.close()
Doing it with an arrow gun directly works:
gunicorn -k aiohttp.worker.GunicornWebWorker -b 0.0.0.0:8000 app:app
But when I try to bind it with a unix socket, I get the following errors.
gunicorn -k aiohttp.worker.GunicornWebWorker -b unix:my_sock.sock app:app
Traceback:
[2015-08-09 12:26:05 -0700] [26898] [INFO] Booting worker with pid: 26898
[2015-08-09 12:26:06 -0700] [26898] [ERROR] Exception in worker process:
Traceback (most recent call last):
File "/home/claire/absapp/venv/lib/python3.4/site- packages/gunicorn/arbiter.py", line 507, in spawn_worker
worker.init_process()
File "/home/claire/absapp/venv/lib/python3.4/site-packages/aiohttp/worker.py", line 28, in init_process
super().init_process()
File "/home/claire/absapp/venv/lib/python3.4/site-packages/gunicorn/workers/base.py", line 124, in init_process
self.run()
File "/home/claire/absapp/venv/lib/python3.4/site-packages/aiohttp/worker.py", line 34, in run
self.loop.run_until_complete(self._runner)
File "/usr/lib/python3.4/asyncio/base_events.py", line 268, in run_until_complete
return future.result()
File "/usr/lib/python3.4/asyncio/futures.py", line 277, in result
raise self._exception
File "/usr/lib/python3.4/asyncio/tasks.py", line 236, in _step
result = next(coro)
File "/home/claire/absapp/venv/lib/python3.4/site-packages/aiohttp/worker.py", line 81, in _run
handler = self.make_handler(self.wsgi, *sock.cfg_addr)
TypeError: make_handler() takes 4 positional arguments but 11 were given
[2015-08-09 12:26:06 -0700] [26898] [INFO] Worker exiting (pid: 26898)
I ran into something on the aiohttp issue ( https://github.com/KeepSafe/aiohttp/issues/136 ) that uses a socket to create a socket that should be put as a parameter in the loop.create_server () function, but I just couldn't get anything to work. (I also don't know if the application in its code is the same web.Application object)
Does anyone know how I can do this? Thank!
source to share
The problem is it GunicornWebWorker
doesn't support unix domain sockets. It comes from GunicornWebWorker.make_handler(self, app, host, port)
who wants the parameters: host
and port
. Obviously you don't have them if you are using a unix socket, but instead have a path to the juice.
Let's look at the beginning GunicornWebWorker._run()
:
def _run(self):
for sock in self.sockets:
handler = self.make_handler(self.wsgi, *sock.cfg_addr)
...
In case -b localhost:8000
sock.cfg_addr
there is ['localhost', 8000]
, but for -b unix:my_sock.sock
it's simple 'my_sock.sock'
. This is where the error occurs TypeError: make_handler() takes 4 positional arguments but 11 were given
. unpacks instead of a list.
A quick way to fix this is by subclassing GunicornWebWorker
and overriding GunicornWebWorker.make_handler()
to ignore host
and port
. They are not used anyway. You can do it like this:
class FixedGunicornWebWorker(worker.GunicornWebWorker):
def make_handler(self, app, *args):
if hasattr(self.cfg, 'debug'):
is_debug = self.cfg.debug
else:
is_debug = self.log.loglevel == logging.DEBUG
return app.make_handler(
logger=self.log,
debug=is_debug,
timeout=self.cfg.timeout,
keep_alive=self.cfg.keepalive,
access_log=self.log.access_log,
access_log_format=self.cfg.access_log_format)
NOTE In yours PYTHONPATH
you will need a fixed worker package. Otherwise Gunicorn will not be able to find him. For example, if you put a fixed worker inside a file fixed_worker.py
inside the same directory that you start from gunicorn
, you can use it like:
$ PYTHONPATH="`pwd`:$PYTHONPATH" gunicorn -k fixed_worker.FixedGunicornWebWorker -b unix:my_sock.sock app:app
UPD Also open an issue in the repository aiohttp
.
source to share