Caveats for scaling a Kafka cluster vertically and horizontally?

We are planning to create a multi-TB Kafka cluster. From the LinkedIn presentations that are supposed to handle the largest Kafka cluster in the world, it looks like they are using several fairly large servers.

We plan to go the other way: start many small Kafka brokers that handle a few GB each.

What are the pros and cons of scaling vertically and horizontally with Kafka? for example, for 50TB, having 5 brokers handling 10TB each, or 5000 brokers handling 10GB each.

These numbers are compiled.

ps: Maintaining 5 or 5000 servers for us has the same operating costs as all automatic ones.

+3


source to share





All Articles