Load balancing with Zookeeper
I am trying to create a load balancer in front of a Zookeeper 3.4.6 cluster. When I do this, the cluster works well, but an exception is thrown:
WARN [NIOServerCxn.Factory: 0.0.0.0/0.0.0.0: 2181: NIOServerCnxn @ 357] - EndOfStreamException encountered: Unable to read additional data from client sessionid 0x0, probable client has a closed socket
org.apache.zookeeper.server .NIOServerCnxn.doIO (NIOServerCnxn.java:228)
org.apache.zookeeper.server.NIOServerCnxnFactory.run (NIOServerCnxnFactory.java:208)
at java.lang.Thread.run (Thread.java:745
This means that Zookeeper understands the Load Balancer as a client and tries to communicate with it. But the load balancer just pings TCP 2181 and exits.
source to share
Are you trying to use a load balancer between your ZooKeeper cluster and clients?
When you pass the ZooKeeper connection string to your clients as multiple endpoints, such as "server1, server2, server3 ...", the clients choose one of the servers and fail over. This way, if all of your clients have the same ZooKeeper endpoint row, you get a balanced pool.
If you install a standard load balancer between the clients and the server, this can lead to such failures. The load balancer doesn't work very well with how ZooKeeper expects its clients to behave. The client must maintain an open TCP connection to the specific server it has a session on by sending periodic heartbeats.
There are certain restrictions on how ZooKeeper clients load the balance themselves (for example, connections won't rebalance if the server reboots), but capturing these limits would require load balancing logic in the ZooKeeper protocol, probably as part of the client implementation.
source to share