Danger of having long running (non-deamon) threads in a Django / Gunicorn application?

I don't need to explicitly use streams in my Django application layer programming (i.e. views) at all. But I noticed a library that looks interesting that handles server-side analytics through streaming.

During Django submission, you should use their Python client to batch send POST requests to the web service on a separate (non-daemon) thread. Typically, I would go with RabbitMQ for something like this, instead of threads, but they wanted to keep the startup costs down for the library.

My question is, are there any disadvantages to this approach? Threads have some extra memory, but I'm not too worried about that. It depends on the number of requests / threads running.

Is the fact that threads are not daemons and are possibly having problems for a long time? My guess is that the Gunicorn process is the main thread of execution and it runs in an infinite loop, so it generally doesn't matter if you have to wait for the non-daemon threads to finish. It is right?

Kind of an open-ended question, but the main one is understanding the impact of non-daemon threads in Django / Gunicorn applications.

+3


source to share


1 answer


Gunicorn uses a pre-fork working model. The master process spawns and manages workflows. For use without using Tornado, there are two kinds of Workers: Sync (default) and Async .

In normal operations, these Workers work in a cycle until the Teacher informs them of a correct completion or kills them. The workers will periodically release a heartbeat to Teacher to show that they are still alive and working. If a timeout heart attack occurs , the DM will kill the Worker and restart him.



Hence, daemon and non-daemon threads that do not interfere with the main Worker loop should not be affected. If a thread is interfering with the main Worker loop, such as a scenario where the thread is doing work, and will provide the results with an HTTP response, then consider using an Async Worker. Async Workers allow the TCP connection to stay alive for long periods of time, while also allowing the Worker to deliver the Master's heartbeat.

+6


source







All Articles