Kafka on AWS ECS, how to handle .host ads without a known instance?
I am trying to get Kafka to work in an AWS ECS container. I have this setup already / working fine in my local docker environment using spotify/kafka
image
To get this working locally, I needed to make sure the ADVERTISED_HOST environment variable was set . ADVERTISED_HOST
needs to be set as an external container for containers, otherwise when I try to connect it just gave me connection refused
.
My local one docker-compose.yaml
has this for the kafka container:
kafka:
image: spotify/kafka
hostname: kafka
environment:
- ADVERTISED_HOST=192.168.0.70
- ADVERTISED_PORT=9092
ports:
- "9092:9092"
- "2181:2181"
restart: always
Now the problem is I don't know what the IP will be as I don't know which instance this will work on. So how do I set this environment variable?
source to share
Your script entry point should call EC2 Metadata Service at startup (in this case http://169.254.169.254/latest/meta-data/local-hostname ) to get the hostname of the external docker and set that variable.
Example:
[ec2-user ~]$ curl http://169.254.169.254/latest/meta-data/local-hostname
ip-10-251-50-12.ec2.internal
source to share