KAFKA consumer and section
1) Can anyone explain how a KAFKA consumer is aware of a partition and consumes data from a specific partition ?.
What is a consumer group?
2) I have 3 partitions and data stored in all partitions,
for example (msg1, msg2, msg3 and msg4). Msg1 → partition 2, msg2 → partition 3, msg3 → partition 1, msg4 → partition 4.
So how can I get the sequential order of messages (msg) from the consumer.
+3
source to share
1 answer
- Brokerage sections are registered with Zookeeper, so by contacting ZK, consumers can find out what the available sections are. The high-level user does this automatically for you.
A consumer group is a common identifier for a group of custom instances, allowing consumption to be parallelized across one or more topics. The number of users concurrently consuming a topic can be less than or equal to the number of topics for that topic, and the topics will be balanced across consumer instances with the same customer group ID.
- Kafka guarantees ordered consumption within a section, but not across sections, so it needs to be handled by the consumer if needed.
+2
source to share