karpenkovarya / airflow_for_beginners Goto Github PK

should be
three_days_ago = today - timedelta(days=3)

should be
two_days_ago = today - timedelta(days=2)

As an additional step, it makes sense to add some parallel tasks to fetch data with other tags.
In this case, we have to take care of the question duplication.

To complete #4 we need to create an HTML template that should be filled with the data later

Create a task that is parsing latest JSON in S3 bucket, and using the result to format an HTML template, that will be sent via email.

DAGs cant see the functions imported from src module

this article might be helpful

Document all steps that were taken to create a pipeline + the reading list, and display it as a README of this repo.

Create a task, that will read questions metadata from the database, filter it by the question score and user reputation and save the result in the JSON format to S3 bucket