Clever Geek Handbook

Difference between Kafka and Kafka Direct?

Is there any difference between kafka and straight kafka. If so, when should you use it instead of the other and what are the benefits?

+3


source to share


2 answers


kafka direct is an optimization that gives better guarantees of fault tolerance and higher reliability of overtime semantics.
park 1.2 introduced by Write Ahead Logs . This ensures that no data received from trusted data sources (i.e. Transactional sources such as Flume, Kafka and Kinesis) is lost due to failures (i.e. Semantics at least once). Even for unreliable (i.e. non-transactional) sources, such as plain old sockets, this minimizes data loss. refer to this link for better understanding



+3


source


New Direct API for Kafka. This allows each Kafka record to be processed exactly once, despite failures, without using Write Ahead Logs. This makes Spark Streaming + Kafka pipelines more efficient while providing higher guarantees of fault tolerance.



0


source






More articles:


All Articles