Creating a Kafka theme in Sarama
3 answers
Can use Sarama to manage themes in Kafka. I am writing a terraform provider to manage Kafka themes and use sarama to do the heavy lifting in the backend.
You have to use the sarama.Broker apis to do this. for example
// Set broker configuration
broker := sarama.NewBroker("localhost:9092")
// Additional configurations. Check sarama doc for more info
config := sarama.NewConfig()
config.Version = sarama.V1_0_0_0
// Open broker connection with configs defined above
broker.Open(config)
// check if the connection was OK
connected, err := broker.Connected()
if err != nil {
log.Print(err.Error())
}
log.Print(connected)
// Setup the Topic details in CreateTopicRequest struct
topic := "blah25s"
topicDetail := &sarama.TopicDetail{}
topicDetail.NumPartitions = int32(1)
topicDetail.ReplicationFactor = int16(1)
topicDetail.ConfigEntries = make(map[string]*string)
topicDetails := make(map[string]*sarama.TopicDetail)
topicDetails[topic] = topicDetail
request := sarama.CreateTopicsRequest{
Timeout: time.Second * 15,
TopicDetails: topicDetails,
}
// Send request to Broker
response, err := broker.CreateTopics(&request)
// handle errors if any
if err != nil {
log.Printf("%#v", &err)
}
t := response.TopicErrors
for key, val := range t {
log.Printf("Key is %s", key)
log.Printf("Value is %#v", val.Err.Error())
log.Printf("Value3 is %#v", val.ErrMsg)
}
log.Printf("the response is %#v", response)
// close connection to broker
broker.Close()
You can take a look at the working code on github . Don't forget to start the kafka broker and import all golang dependencies before running the code.
+4
source to share
Better to use: https://github.com/Shopify/sarama/blob/master/admin.go instead of connecting directly to the broker.
This handles a lot of cases like:
- You can add multiple broker addresses for cluster configuration.
- Determining which broker acts as a controller is done automatically.
+2
source to share
Indeed, in newer versions of Sarama, you can use ClusterAdmin to create themes. Below you can find some sample code:
package main import ( "github.com/Shopify/sarama" // Sarama 1.22.0 "log" ) func main() { brokerAddrs := []string{"localhost:9092"} config := sarama.NewConfig() config.Version = sarama.V2_1_0_0 admin, err := sarama.NewClusterAdmin(brokerAddrs, config) if err != nil { log.Fatal("Error while creating cluster admin: ", err.Error()) } defer func() { _ = admin.Close() }() err = admin.CreateTopic("topic.test.1", &sarama.TopicDetail{ NumPartitions: 1, ReplicationFactor: 1, }, false) if err != nil { log.Fatal("Error while creating topic: ", err.Error()) } }
0
source to share