Why is Kafka's performance slow?
I have one simple theme and one simple Kafka consumer and producer using the default configuration.
The program is very simple, I have two threads.
In producer, it keeps on sending 16 byte data.
And on the consumer side, he continues to receive.
I found the fact that the throughput for the manufacturer is about 10 MB / s is ok.
But the bandwidth for the consumer is only 0.2MB / s. I've disabled all debug logs, but that doesn't make it any better. The test is performed on the local machine. Does any body have an idea of ββwhat is going wrong? Thank!
The code I used is below: Manufacturer:
KafkaProducer producer = new KafkaProducer(props);
int size = 16;
byte[] payload = new byte[size];
String key = "key";
Arrays.fill(payload, (byte) 1);
ProducerRecord record = new ProducerRecord("test",0,key.getBytes(),payload);
while(true){
producer.send(record);
}
Consumer:
Properties consumerProps = new Properties();
consumerProps.put("zookeeper.connect", "localhost:2181");
consumerProps.put("group.id", "test");
ConsumerConnector consumer = Consumer.createJavaConsumerConnector(new ConsumerConfig(consumerProps));
Map<String, Integer> topicCountMap = new HashMap<String, Integer>();
topicCountMap.put("test", 1);
Map<String, List<KafkaStream<byte[], byte[]>>> consumerMap = consumer.createMessageStreams(topicCountMap);
List<KafkaStream<byte[], byte[]>> streams = consumerMap.get("test");
ConsumerIterator<byte[], byte[]> it = streams.get(0).iterator();
while(it.hasNext()){
it.next().message();
}
+3
source to share
1 answer