What is the use of advanced master pick algorithms over bullies algorithm?
I have read how the current top election algorithms like Raft, Paxos or Zab elect the cluster master and cannot understand why they use complex algorithms instead of the simple bully algorithm.
I am developing a cluster library and am using UDP multicast for heartbeat messages. Each node concatenates a multicast address and periodically sends datagram packets to that address. If the nodes find out there is a new node that is sending packets to that multicast, the node is just added to the cluster and similarly, when the nodes in the cluster do not receive any packet from the node, they remove it from the cluster. When I need to select a master node, I just iterate over the nodes in the cluster and select the oldest one.
I have read several articles that suggest that this approach is ineffective and more sophisticated algorithms such as Paxos should be used to select a master or to detect failures via heartbeat messages. I couldn't figure out why Paxos is better suited for split-brain scenarios or other network outages than the traditional bully algorithm, because I can easily tell when the node quorum leaves the cluster without using Raft. The only advantage I see is the number of packets that each server has to handle; only the master sends heartbeat messages to Raft, while in this case each node has to send heartbeat messages to each other. However, I don't think this is a problem as I can simply implement a similar beat algorithm without changing the main pick algorithm.
Can anyone clarify this?
source to share
From a theoretical point of view, Raft, Paxos and Zab are not leader selection algorithms. They solve another problem called consensus.
In your specific scenario, the difference is as follows. With a leader selection algorithm, you can ensure that ultimately one node is the leader. This means that for some time several nodes may think that they are leaders and therefore may act as one. In contrast, with the consensus algorithms described above, you can ensure that at most one leader exists during a logical moment.
The consequence of this. If the security of the system depends on the presence of one leader, you may find it difficult to rely only on the election of a leader. Let's look at an example. Nodes receive messages from UDP multicast and do A if the sender is the leader, but do B if the sender is not the leader. If two nodes check the oldest node in the cluster at several different points in time, they may see different leaders. These two nodes can then receive the multicast message and process it in different ways, perhaps violating some of the security properties of the system you would like to hold (for example, so that all nodes do A or B, but never do A and the other B).
With Raft, Paxos and Zab, you can overcome this problem as these algorithms create a kind of logical epochs, each with at most one leader.
Two entries here. First, the bully algorithm is defined for synchronous systems. If you do implement it as described in García-Molina's article, I believe you might have problems with your partially synchronous system. Second, the Zab algorithm relies on a kind of bully algorithm for asynchronous systems. The leader is selected by comparing the length of their stories (which minimizes system recovery times). Once the leader is elected, he tries to start the Zab protocol, which in turn blocks the epoch for the leader.
source to share