Multithreading with websockets
It's a design issue. I have the following implementation
Multiple client connections -----> Server ------> Matching DB connections
Communication between client and server is done using network sockets. It is a single threaded application. Obviously, this project does not scale as the server load is too high and it takes too long to respond back to clients. Backward operations involve processing large amounts of data.
My question is, is it a good idea to create a new thread for every websocket connection? This implies 500 threads for 500 clients (the number of network sockets will be the same, be it multi-threaded or single-threaded). This would ease the load on the server and therefore make life easier.
or
Is there any better logic to achieve scalability? One of them can create threads at the merit of the job, and the rest can be processed by the main thread. It somehow seems to be returning to the same problem again in the future.
Any help here would be greatly appreciated.
source to share
There are two approaches to this problem
- one thread per request
- a fixed number of threads to manage all requests
You are actually using the second approach, but using only 1 thread.
You can improve it by using a thread pool to handle your requests instead of just one.
The number of threads used for the second approach depends on your application. If you have high CPU usage and a certain number of long I / O operations (read or write to disk or network), you can increase that number.
If you don't have I / O, the number of threads should be closer to the number of processor cores.
Note. existing web servers use these two approaches for HTTP requests. As an example Apache uses the first (one thread for one request) and NodeJs uses the second (it's event driven).
In any case, use a timeout system to unblock very long requests before the server crashes.
source to share
You can see two very good scalable web server: the Apache and Node.js .
Apache , running in multi-threaded (production) mode, will create new threads for new connections (note that requests from the same browser are served from the same thread via keep-alive).
Node.js is vastly different and uses an asynchronous workflow to delegate tasks.
Hence Apache scales very well for computationally intensive tasks, whereas Node.js scales well for many (huge) small event-based requests.
You note that on the backend you are doing some heavy lifting. This means that you must create multiple threads. How? Create a queue of threads with a limit MAX_THREADS
and a prefix MAX_THREADS_PER_CLIENT
, serving duplicate client requests using the same thread. Your main thread should spawn new threads.
If you can, you can include some good Node.js features. If some task in a thread is taking too long, kill that thread with a callback for the job to create a new one when the job is done. You can do a test to even train NN to know when to do it!
Take a blast!
source to share