data-engineering-project's Introduction

data-engineering-project

Project in which we practice with data engineering exercices.

Branching strategy

The branching strategy we follow is Gitflow: https://nvie.com/posts/a-successful-git-branching-model/

Branch name convention

DEP-XYZ-Name

Where DEP is the code of the project XYZ is the number of the issue and Name is a description(sometimes summarized version) of the name of the issue.

Commit message convention

Commit convention: https://www.conventionalcommits.org/en/v1.0.0/

Examples:

feat: allow provided config object to extend other configs BREAKING CHANGE: extends key in config file is now used for extending other config files feat!: send an email to the customer when a product is shipped feat(api)!: send an email to the customer when a product is shipped

data-engineering-project's People

Contributors

Watchers

data-engineering-project's Issues

Add structure to the Python repository

We need to stablish a proper structure to the repository. We need a src directory for the code, a test directory for the tests, doc for documentation and so on.

Create SQL to insert data into the first reporting table

TBD (To be determined)

Create reporting table in SQL

As a software engineer I need to define the first reporting table in SQL.

This table needs to contain several aggregations of the data in the source database to create new fields. It is important to add two fields for the partitioning of the date. These would be "ingestion_date", "ingestion_timestamp".

Add information about project in README

We would need to link this project to the README so that the people involved can easily locate it. Also explain the meaning of the code for the project that we will use for our branches.

DEP-XYZ-Name. DEP is the code of the project, XYZ is the number of the issue and Name has to be a descriptive and sometimes summarized version of the name of the issue.

Design risk database

As a business analyst I need a database designed around the products of a bank that sells products for car insurance. It is important to store information about the customer, the products and the risk of a customer for future loans.

Aceptance criteria: