A demo of a self-contained dbt project for transforming COVID-19 data from a public source.
Dbt Demo is an example of a software project that uses the command-line
interface (CLI) tool dbt
to transform raw data into analytics-ready data
models. It began at version v0.1.0 with a set of starter files generated using
the command dbt init
. Since then, it has iteratively evolved to depict what a
dbt could look like for a fictional e-commerce business: Jaffle Shop.
What this repo is:
- A self-contained playground dbt project, useful for testing scripts and communicating some of the core dbt concepts.
- A demonstration of some best practices, such as:
- Dbt's standard file naming patterns (which make more sense on larger projects, rather than this five-model project)
- Simple CI/CD integrations
What this repo is not:
- A tutorial — check out the Getting Started Tutorial for that. Notably, this repo contains some anti-patterns to make it self-contained, namely the use of seeds instead of sources.
- A demonstration of other best practices — check out the dbt Learn Demo repo
instead. We want to keep this project as simple as possible. As such, we chose
not to implement:
- A merge request flow
- More complex CI/CD integrations
- A demonstration of using dbt for a high-complex project, or a demo of advanced features (e.g. macros, packages, hooks, operations) — we're just trying to keep things simple here!
TBD
Dbt Demo supports UNIX-flavored operating systems, including:
It also supports the Windows operating system when used with Microsoft's native Windows Subsystem for Linux (WSL) or a 3rd-party emulator like Git BASH (part of Git for Windows).
To use or test Dbt Demo, the following software must first be installed on your system:
- Python 3.10.9 or higher
To contribute to Dbt Demo's development and maintenance, the following DevOps toolchain must also be installed:
- Git 2.40.1 or higher
- GitFlow 0.4.1_1 or higher
- pre-commit v3.2.1 or higher
Try running the following commands to see dbt
in action:
dbt clean
dbt debug
dbt seed
dbt run
dbt test
dbt debug
dbt docs generate
dbt docs serve
For CI/CD purposes, try running this command:
pre-commit run --all-file
Documentation for Dbt Demo is this README itself, plus those listed in the subsections that follow.
Currently, there are no known issues.
- Learn more about dbt in the docs
- Check out Discourse for commonly asked questions and answers
- Join the chat on Slack for live discussions and support
- Find dbt events near you
- Check out the blog for the latest news on dbt's development and best practices
API documentation, tutorials, and other online references and made portions of this assignment possible. See REFERENCES.md for a list of some.