A plugin for Airflow dags and tasks that sets up Sentry for error logging.
Install the sentry-sdk
:
$ pip install sentry-sdk
Create a plugin folder in your AIRFLOW_HOME
directory if you do not have one yet:
$ mkdir -p $AIRFLOW_HOME/plugins
Then clone this repository in there:
$ cd $AIRFLOW_HOME/plugins
$ git clone [email protected]:getsentry/sentry-airflow.git
Make sure you have setup your SENTRY_DSN
in your environment variables! The DSN can be found in Sentry by navigating to [Project Name] -> Project Settings -> Client Keys (DSN). Its template resembles the following: '{PROTOCOL}://{PUBLIC_KEY}@{HOST}/{PROJECT_ID}'
Install the sentry-sdk
into Google Composer's Python dependencies.
Add this folder to your plugin directory:
$ gcloud composer environments storage plugins import --environment ENVIRONMENT_NAME \
--location LOCATION \
--source PATH_TO_LOCAL_FILE \
--destination PATH_IN_SUBFOLDER
(For more information checkout Google's Docs)
Either set an environment variable on Google composer for your SENTRY_DSN
or in the Airflow webserver UI, add a connection (Admin->Connections) for sentry_dsn
. Let the connection type be HTTP
and the host be the DSN value.