Clever Geek Handbook

How to view a kafka post

Is it possible to somehow view the content of a message sent by kafka on a given topic? Say something like looking at the last 5 posts on this topic if possible.

+27


source to share


7 replies


You can use the console consumer to view messages generated on any topic:



bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
+52


source


Use the Kafka consumer provided by Kafka:

bin/kafka-console-consumer.sh --bootstrap-server BROKERS --topic TOPIC_NAME


It will display messages when it receives them. Add --from-beginning

if you want to start from the beginning.

+3


source


If you are doing from Windows folder I mean if you are using Kafka from Windows machine

kafka-console-consumer.bat --bootstrap-server localhost:9092 --<topic-name> test --from-beginning
+1


source


The Kafka toolkit is decently discussed in the following SO threads, which are fairly comprehensive in scope:

If you want a web-based tool, try Kafdrop 3 (this is a resurrection of the original Kafdrop, which is mostly inactive right now). It allows you to browse topics and configure the cluster (but it is not a complete admin tool). This is a simple spring boot application and comes with a Docker build. (Small disclaimer: I am one of the authors, but I am not the original author.)

For a command line tool, try Kafkacat for viewing topics and posting messages. At the time of writing, it also supports printing message headers (as opposed to the built-in Kafka tools).

+1


source


If you want to do this programmatically, you can write an application that wraps the Kafka client (it will be available for almost every language), or a bash script that uses either one of the built-in kafka kafka-console-consumer

or Kafkacat tools, which is a little more flexible (but the downside is that you need to download a separate tool while kafka-console-consumer

Kafka comes with).

If you need a GUI that displays the last few posts on a topic, you can use Kafdrop 3 or the Kafka Tool . The first is a web application (Springboot application) and the second is a Swing based desktop application.

+1


source


You can try Kafka Magic - it's free, and you can write complex queries in JavaScript referencing message properties and metadata. Also works with Avro serialization

0


source


If you are looking for an easy and intuitive way to view and search Apache Kafka messages, you should try KaDeck . The public version is completely free and supports Win, Mac OS and Linux.

An enterprise version is also available for use in a corporate context, which includes a web service.

-1


source






More articles:


All Articles