Airflow will continue to show approximate bundles even after being removed from the configuration
The airflow examples remain in the UI even after I turned it off load_examples = False
in the config file.
The system reports that there are no shortcuts in the shortcuts folder, but they remain in the UI because the scheduler marked it as active in the metadata database.
I know that one way to delete them would be to directly delete those rows in the database, but that is of course not ideal. How do I proceed to remove these packages from the UI?
source to share
Let's assume you have set up airflow through Anaconda. Take another look at the airflow in the python packages folder and follow below.
After following the instructions fooobar.com/questions/560500 / ...
- Go to $ AIRFLOW_HOME / lib / python2.7 / site-packages / airflow directory
- Delete the directory named example_dags, or simply rename it to go back.
- Reboot the web server
cat $ AIRFLOW_HOME / airflow-webserver.pid | xargs kill -9
airflow web server -p [port number]
source to share
Definitely airflow resetdb
works here.
I am creating several shell scripts for various purposes like starting the webserver, starting the scheduler, updating the dag, etc. I only need to run the script to accomplish what I want. Here's a list:
(venv) (base) [pchoix@hadoop02 airflow]$ cat refresh_airflow_dags.sh
#!/bin/bash
cd ~
source venv/bin/activate
airflow resetdb
(venv) (base) [pchoix@hadoop02 airflow]$ cat start_airflow_scheduler.sh
#!/bin/bash
cd /home/pchoix
source venv/bin/activate
cd airflow
nohup airflow scheduler >> "logs/schd/$(date +'%Y%m%d%I%M%p').log" &
(venv) (base) [pchoix@hadoop02 airflow]$ cat start_airflow_webserver.sh
#!/bin/bash
cd /home/pchoix
source venv/bin/activate
cd airflow
nohup airflow webserver >> "logs/web/$(date +'%Y%m%d%I%M%p').log" &
(venv) (base) [pchoix@hadoop02 airflow]$ cat start_airflow.sh
#!/bin/bash
cd /home/pchoix
source venv/bin/activate
cd airflow
nohup airflow webserver >> "logs/web/$(date +'%Y%m%d%I%M%p').log" &
nohup airflow scheduler >> "logs/schd/$(date +'%Y%m%d%I%M%p').log" &
Don't forget chmod +x
for these scripts
I hope you find this helps.
source to share
Air flow 1.10+:
- Edit airflow.cfg and install
load_examples = False
- For each dag example run the command
airflow delete_dag example_dag_to_delete
This avoids dumping all airflow in dB.
(Since Airflow 1.10 has a command to remove dag from the database, see this answer )
source to share