This repo contains an Astronomer project with multiple example DAGs showing how to use Airflow for SQL use cases. A guide discussing the DAGs and concepts in depth can be found here.
This tutorial has five DAGs showing how to accomplish the following SQL use cases:
- Executing two dependent queries
- Executing a paramterized query
- Loading data into a database
- Executing an ETL pipeline with transformations in Pandas
- Executing a query using dag-factory
The easiest way to run these example DAGs is to use the Astronomer CLI to get an Airflow instance up and running locally:
- Install the Astronomer CLI
- Clone this repo somewhere locally and navigate to it in your terminal
- Initialize an Astronomer project by running
astro dev init
- Start Airflow locally by running
astro dev start
- Navigate to localhost:8080 in your browser and you should see the tutorial DAGs there