ZeroMQ push / pull model
I decided to write some test code to find out how the pusher works - a lot of pullers, and my suspicions were justified.
The scavengers receive messages to be linked, for example, the first message is received with the 1st stripper, the second by the 2nd, etc. I simulated a situation where one of the pullers stayed busy after receiving the message, but when he arrived in time to receive the message, it was still queued, so I lost the message. This is bad. I want this message to be received by the next "free" stripper. It's real?
My test code. I am using zmqpp as bindings
void main()
{
auto _socket = sIpcContext->CreateNewSocket(zmqpp::socket_type::push);
_socket->bind("tcp://*:4242");
for (auto i = 0; i < 3; ++i)
{
new std::thread([&](int _idx)
{
auto idx = _idx;
auto sock = sIpcContext->CreateNewSocket(zmqpp::socket_type::pull);
sock->connect("tcp://127.0.0.1:4242");
for (;;)
{
std::string msg;
sock->receive(msg);
std::cout << idx << " received: " << msg << std::endl;
if (idx == 1)
{
std::cout << "Puller 1 is now busy" << std::endl;
std::this_thread::sleep_for(std::chrono::seconds(10000));
}
}
}, i);
}
for (auto i = 0;; ++i)
{
_socket->send(std::to_string(i));
std::this_thread::sleep_for(std::chrono::seconds(1));
}
}
I am getting this output:
0 received: 0
0 received: 1
1 received: 2
Puller 1 is now busy
2 received: 3
0 received: 4
2 received: 6
0 received: 7
2 received: 9
0 received: 10
2 received: 12
0 received: 13
2 received: 15
As you can see, 5, 8, etc. "missed", but actually queued in stripper # 1
source to share
The 0MQ manual explains this case (calls it the post branch counterpart):
This is a post branch analogy. If you have one queue per meter and have people buying stamps (fast, simple transaction) and some people opening new accounts (very slow transaction), then you will find brand buyers unfairly stuck in queues. As with mail, if your messaging system is unfair, people will get annoyed.
The post office solution is to create one queue so that even if one or two counters get stuck in slow operation, the other counters will continue to serve customers on a first come, first served basis.
In short, you should use ROUTER when dealing with slow workers.
source to share