How do I create a server that can handle 20,000 consistent connections?
I find it difficult to elaborate on exactly what the server needs to do (because of the NDA and what not), but it should suffice to say that it needs to handle a lightweight binary protocol with many concurrent users connected, ~ 20.000 is where we have quite a decent estimate.
Please note that clients will not send / receive all the time, but I need to keep the socket open because when the client needs a response, we need it as quickly as possible and have no time for the overhead of opening a new connection every time.
The protocol is very light, so there won't be a lot of data on the wire - the main problem is to open 20,000 sockets at the same time.
(I know the specs are a little fuzzy, but I really can't go into details)
I have a pretty decent idea of ββwhat I need to do and what kind of hardware we need for the server (s), but I figured I was asking existing projects, technologies, languages ββ(like Erlang), etc. here. that might help me create this.
How can this be achieved?
source to share
If you don't need to go through a firewall, consider using a UDP based protocol. NFS is a good example of a UDP based protocol. UDP has no TCP setup overhead and can scale to over 65K concurrent connections. However, if you want guaranteed delivery, you will need to create this functionality in your application.
For performance with large user bases, consider using a non-blocking I / O based server architecture.
Another subject worth looking into is Douglas Schmidt's Adaptive Communication Environment (ACE). It is a mature C ++ platform for building high-performance servers, mainly targeting telecom applications. It supports many streaming models and handles most of the hard stuff for you. You might find that the time spent learning how to manage it will be saved with less debugging effort for messy sync issues.
source to share
Maintaining 20,000 connected sockets is not a problem. You can do this with C on Windows (Server) quite easily as long as you are using the I / O completion ports and / or threadpool APIs.
The real prblem, I think, is generating data for those 20,000 connections. This may require some exotic solutions - Erlang or something else. But the socket side is not trivial, but fine within the traditional service design.
source to share
Take a look at CCR from the robots guys at Microsoft. I let you program the Erlang type (message passing, queues, etc.), but just using C # and not a completely new functional language.
Plus, it can use an asynchronous programming model where you don't need dozens of threads in thread pools to do your stuff. It is much faster and produces really elegant code.
I myself use it for an SMS server that has to spit out SMS at ridiculous speeds and it does it without stressing the processor at all
source to share
Erlang with its lightweight streams and awesome binary processing will make it great. In terms of hardware, I don't see that you would need an extremely expensive server if the protocol is very light, but that will depend on other processing to be done after the packet has been received.
Edit
If you need to do index lookups or something, Mnesia is also larger and supports both in memory and disk space and is fully allocated if you need to migrate to more servers.
Some real world info on Erlangs' connection handling capabilities http://www.sics.se/~joe/apachevsyaws.html
source to share