What is it
Airflow manages and monitors scheduled workflows such as data pipelines. It is configured through a Python SDK and in GCP it's hosted in a Cloud Composer environment.
When to use it
When you want a GUI to monitor scheduling and execution of workflows. However, if a workflow is a batch data pipeline that is just a SQL statement, consider using Dataform instead.
How to learn it
Why it's on hold
Airflow was assessed in early stages of data pipeline development at Einride, but dbt has since become the preferred toolchain for developing and orchestrating data pipelines.