Why Docker Overhead Networks Require Consensus?
Just spent my day reading on Docker overlay networks, very cool stuff. I just can't find the answer to one thing.
According to the docs:
- If you install and use Docker Swarm, you automatically get overlay networks on your managers / worker hosts and don't need to configure anything more; but ...
- If you just want to have an overlay network (non-Swarm) on multiple hosts, you need to configure this network with an external "KV Store" (consensus server) such as Consul or ZooKeeper
I'm wondering: why?!? Obviously, overlay networks require consensus among peers, but I'm not sure why or who these peers even are.
And I'm just guessing there is an internal / consensus server with content coming out of the box with Swarm. Yes? Not? Thanks in advance!
source to share
Swarm mode uses Raft to reach manager consensus with built-in KV storage. Prior to swarm mode, it was possible to overlay network networks using third-party KV stores. The overlay network itself does not require consensus, it just relies on what the KV store says independently of other nodes or even because of its local state (I found this with difficulty). KV stores usually set a consensus for HA.
KV store keeps track of the distribution of IP addresses in containers running on each host (IPAM). This allows docker to only allocate a given address once and know which docker host it should bind to when you connect to a container running on a different host. This should be external from any docker host and preferably in a HA configuration (e.g. swarm mode negotiation) so that it can continue to work even if some docker nodes are down.
The overlay of networks between docker nodes is only associated with nodes that have containers in that overlay network. Therefore, when an IP address is allocated and discovered, all messages only happen between hosts with matching containers. This is easy to see with the swarm mode, if you create a network and then list the networks in the workplace, it won't be there. After a container in this network is assigned, a network appears. With docker, this reduces the overhead of multiple host networks and also improves the security of the architecture. The result looks like this:
The raft summit is only required for the election of leaders. Once a node is selected by the leader and enough nodes remain to reach consensus, only one node writes to the KV store and maintains the current state. All others are followers. This animation describes it better than I ever could .
Finally, you don't need to configure external KV storage to use overlay networks outside of swarm mode services. You can implement swarm mode, set up overlay networks with the option, --attachable
and run containers outside of swarm mode on that network, just like with external KV storage. I've used this in the past as a transient to get containers into swarm mode, where some were done with docker-compose and others were deployed like a swarm stitch.
source to share