Clever Geek Handbook

Can one task be used in multiple DAGS in an airflow?

I have a task_a that I want to use in DAG_1 and DAG_2. Is this possible in an air stream?

task_a = SomeOperator(
task_id='some_id',
bash_command='some_command',
#instead of just
dag= DAG_1 # I want to assign this task to multiple dags
#dag=assign_multiple_dags_here(DAG_1 and DAG_2)
)

Is it possible?

+3


source to share


2 answers


you can always do something with partial

and then assign it to two different entries:

from functools import partial
task_template = partial(SomeOperator, some_id='id', some_command='cmd')
task_template(dag=dag1)
task_template(dag=dag2)


you can also just create a function that does this:

def create_task(dag):
    return SomeOperator(some_id='id', some_command='cmd', dag=dag)

for d in (dag1, dag2):
    create_task(d)
+2


source


In accordance with this project No.

Tasks are part of the DAG. Each DAG run creates a task instance.



This is necessary to simplify the work with the frame.

+2


source






More articles:


All Articles