How to view a kafka post
Is it possible to somehow view the content of a message sent by kafka on a given topic? Say something like looking at the last 5 posts on this topic if possible.
You can use the console consumer to view messages generated on any topic:
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
Use the Kafka consumer provided by Kafka:
bin/kafka-console-consumer.sh --bootstrap-server BROKERS --topic TOPIC_NAME
It will display messages when it receives them. Add --from-beginning
if you want to start from the beginning.
If you are doing from Windows folder I mean if you are using Kafka from Windows machine
kafka-console-consumer.bat --bootstrap-server localhost:9092 --<topic-name> test --from-beginning
The Kafka toolkit is decently discussed in the following SO threads, which are fairly comprehensive in scope:
- Kafka has some kind of default web interface
- Kafka theme viewer?
If you want a web-based tool, try Kafdrop 3 (this is a resurrection of the original Kafdrop, which is mostly inactive right now). It allows you to browse topics and configure the cluster (but it is not a complete admin tool). This is a simple spring boot application and comes with a Docker build. (Small disclaimer: I am one of the authors, but I am not the original author.)
For a command line tool, try Kafkacat for viewing topics and posting messages. At the time of writing, it also supports printing message headers (as opposed to the built-in Kafka tools).
If you want to do this programmatically, you can write an application that wraps the Kafka client (it will be available for almost every language), or a bash script that uses either one of the built-in kafka kafka-console-consumer
or Kafkacat tools, which is a little more flexible (but the downside is that you need to download a separate tool while kafka-console-consumer
Kafka comes with).
If you need a GUI that displays the last few posts on a topic, you can use Kafdrop 3 or the Kafka Tool . The first is a web application (Springboot application) and the second is a Swing based desktop application.
You can try Kafka Magic - it's free, and you can write complex queries in JavaScript referencing message properties and metadata. Also works with Avro serialization
If you are looking for an easy and intuitive way to view and search Apache Kafka messages, you should try KaDeck . The public version is completely free and supports Win, Mac OS and Linux.
An enterprise version is also available for use in a corporate context, which includes a web service.