Consumer: Cannot connect to amqp: // user: ** @localhost: 5672 //: [Errno 111] Connection failed
I am trying to build my airflow using docker and rabbitMQ. I am using rabbitmq: 3-management image. And I can access rabbitMQ UI and API.
In airflow, I create an airflow webserver, airflow scheduler, work air, and airflow flower. The Airflow.cfg file is used to adjust the airflow.
Where do I use broker_url = amqp://user:password@127.0.0.1:5672/
andcelery_result_backend = amqp://user:password@127.0.0.1:5672/
My docker compose file looks like this
version: '3'
services:
rabbit1:
image: "rabbitmq:3-management"
hostname: "rabbit1"
environment:
RABBITMQ_ERLANG_COOKIE: "SWQOKODSQALRPCLNMEQG"
RABBITMQ_DEFAULT_USER: "user"
RABBITMQ_DEFAULT_PASS: "password"
RABBITMQ_DEFAULT_VHOST: "/"
ports:
- "5672:5672"
- "15672:15672"
labels:
NAME: "rabbitmq1"
webserver:
build: "airflow/"
hostname: "webserver"
restart: always
environment:
- EXECUTOR=Celery
ports:
- "8080:8080"
depends_on:
- rabbit1
command: webserver
scheduler:
build: "airflow/"
hostname: "scheduler"
restart: always
environment:
- EXECUTOR=Celery
depends_on:
- webserver
- flower
- worker
command: scheduler
worker:
build: "airflow/"
hostname: "worker"
restart: always
depends_on:
- webserver
environment:
- EXECUTOR=Celery
command: worker
flower:
build: "airflow/"
hostname: "flower"
restart: always
environment:
- EXECUTOR=Celery
ports:
- "5555:5555"
depends_on:
- rabbit1
- webserver
- worker
command: flower
I can create images using docker compose. However, I am unable to connect my airflow scheduler to rabbitMQ. I am getting the following error:
consumer: unable to connect to amqp: // user: ** @localhost: 5672 //: [Errno 111] Connection refused.
I tried using 127.0.0.1 and localhost both.
What am I doing wrong?
source to share
In containers, airflow
you can connect to the service rabbit1
. So all you have to do is change amqp://user:**@localhost:5672//:
to amqp://user:**@rabbit1:5672//:
and it should work.
Docker compose creates a default network and connects services that do not explicitly define a network for it.
You don't need to open ports 5672 and 15672 on rabbit1 unless you want to be able to access it from outside the application.
Also, it is generally not recommended to create images inside docker-compose.
source to share