Difference between dcos-kafka-service and meso-kafka
I am doing a POC to deploy Kafka as an application on a Mesos cluster. I came across these two codes on github. One was developed by apache-mesos ( github page ) and the other was developed by mesosphere and can only run on DCOS ( github page ).
Q: I would like to know if there are any differences between DCOS-Kafka and mesos-Kafka in terms of features and advanced functionality.
Regarding Mesos-Kafka: In recent months, I don't see active engagement on github (and some open issues) for meso-kafka. Can I assume the service is robust enough that I can use it in a production environment? Any inputs to this would be helpful.
source to share
kakfa-mesos is a package that includes the release of Kafka and a custom mesos scheduler that addresses the issues of running Kafka as a stateful service on a marathon. I think post , but confluent is helpful. It also includes a RESTful api for performing ops tasks and aims to enable these features in the future (this was pulled from the article I linked)
- Integrate Kafka commands (like kafka-themes, etc.) into Planner so that it can be used via CLI and REST APIs.
- Auto-scaling of clusters (including auto-partitioning) so that resources (CPU, RAM, etc.) that are used by brokers can be used elsewhere in known traffic valleys.
- Assigning partitions in a rack for fault tolerance.
- Hooks so that producers and consumers can also be run from the Scheduler and managed by the cluster.
- Automatic partition reassignment based on load and traffic
I haven't used it in a production environment myself, but it does have Confluent support, which is a good sign.
DC / OS Kafka, on the other hand, is a DC / OS service that will probably only be useful if you are already using or planning to run services through the DC / OS Mesosphere. It also includes an API and CLI management tool, but is less ambitious with additional features. The current feature set includes
- Single command setup for fast provisioning
- Multiple clusters for multiple leases with DC / OS
- HA software configuration and updates
- Storage volumes to improve data longevity, known as Mesos Dynamic * * Disclaimer and fixed volumes
- Integration with syslog-compatible logging services for diagnostics and troubleshooting
- Integration with statsd compliant statistics services to monitor performance and performance
source to share