How to do Fields Grouping in Kafka Spout (JAVA)?

- I have a requirement when I use delta updates of items from Kafka queue sections.

  • The manufacturer guarantees that all delta updates corresponding to a given item are available in the same section and in order. Example: I1 and I2 are elements. There updates say that I1-Up1, I1-Up2 will be in the same section. Likewise, I2-Up1 and I2-Up2 will be in the same partition.

  • Hence, when my first set of Bolts gets it, they should get fields grouped based on elements (I1 and I2). I mean the same bolt should be getting all I1 updates ok.

  • The problem is how can I specify this in Bolt right after the Spout, since KafkaSpout has to do something.

  • A quick search said that I can write my own Kafka schema (implement MultiScheme), but there are not enough code examples. I wrote one but not sure how to make sure this is a field grouping.

+3


source to share





All Articles