chungeun-choi / airflow_deploy Goto Github PK

This repository is defined to deploy Airflow using Docker containers in a dynamic infrastructure environment.

When deploying Airflow, the worker component is separated into its own deployment. The remaining components, including webserver, scheduler, database, and Redis, are defined in the same YAML file. The AIRFLOW__WEBSERVER__SECRET_KEY option allows fetching worker logs from the webserver for monitoring.

• The provided definition files use a requirements.txt file and Dockerfile to build Docker images and create containers for Airflow services to support the required modules in workflows (refer to the image).

  

• To enable communication between the Airflow worker service and the Airflow webserver, create a .env file and define the AIRFLOW__WEBSERVER__SECRET_KEY value.

• Requires Docker Engine to be installed.

• TLS settings are not applied for quick deployments.