Python + uwsgi - multiprocessing and general application state
We have a flash application running behind uwsgi with 4 processes. Its an API that serves data from one of our two ElasticSearch clusters.
When the application is loaded, each process fetches the configuration from the external database to check which ES cluster is active and connect to it.
Evey gets a POST request from time to time (from aws SNS service) that informs all clients about the ES cluster switch. This triggers the same function as in the bootstrap-pull config from DB to connect to the active ES cluster.
It works well as a single process, but when we have more than one process, only one of them will get updated (the one receiving the POST request) ... where other processes are still connected to the inactive cluster.
Disabling the configuration of each request to make sure the ES cluster we are using is active will slow down. Im thinking to install redis locally and store active_es_cluster there ... any other ideas?
source to share
I think there are two paths you could take.
-
You have a / set _es_cluster endpoint that gets caught in your SNS POST request. This endpoint then sets the "active_es_cluster" key, which is read on every ES request by your other processes. The downside to this is that on every ES request, you need to search for redis first.
-
You have a separate process that receives a POST request (I'm assuming the clusters don't change often). The purpose of this process is to receive a post request and only uWSGI will gracefully restart your other processes in flasks.
The advantages of the second option:
- No need to remove redis on every request
- Let uWSGI handle the reboots for you (which is good)
- You have already set up a runtime configuration setting so it should "work" with your existing application
source to share