This documentation describes how to use the Concha API using production deployment, Postman, or cURL.
When you make a request to the REST API, you will specify an HTTP method and a path that follow a specific schema. Additionally, you might also specify request headers and path, query, or body parameters (Ex. Authorization info in header for protected routes). The API will return the response status code, response headers, and potentially a response body.
01. Scope of Project
02. Project Structure
03. Run Project Locally
04. Run on GCP
Assignment Document : GoogleDoc
Notion Doc: NotionDoc
Postman Collection : Postman
Stack | Technology |
---|---|
Backend | FastAPI |
Database | PSQL |
Database Migration | Alembic |
O.R.M | SQLAlchemy |
API Platform | Postman |
Authorization, Authentication | Bcrypt, JWT |
Version Control | Git |
Application Devlopment | Docker, Docker Compose |
Cloud | GCP |
<project-root>
βΒ
βββ server # "app" (python package) for BE app
βΒ Β βββ __init__.py # this file makes "app" a "Python package"
βΒ Β βββ main.py # "main" module, e.g. import app.main
βΒ Β βββ dependencies.py # "dependencies" module, e.g. import app.dependencies
βΒ Β βββ dependencies.py # "dependencies" module, e.g. import app.dependencies
βΒ β
βΒ Β βββ routers # "routers" is a "Python subpackage"
βΒ Β βββ __init__.py # makes "routers" a "Python subpackage"
βΒ Β βββ users.py # "users" submodule, e.g. import app.routers.items
βΒ Β βββ audio_data.py # "audio_data" submodule, e.g. import app.routers.users
βΒ
βββ psql_db # "app" is a Python package
βΒ Β βββ __init__.py # this file makes "app" a "Python package"
βΒ
βββ alembic # Datbase migrations
βΒ Β βββ versions
βΒ Β βββ env.py
βΒ
βββ tests # tests package
βΒ Β βββ database.py # database settings for running tests
βΒ Β βββ test_user.py # tests for /user route
βΒ Β βββ test_audio.py # tests for /audio-data route
βΒ
βββ Dockerfile # Dockerfile to build docker image for the project
βΒ
βββ docker-compose-dev.yml # Docker compose file for dev environment
βΒ
βββ docker-compose-prod.yml # Docker compose file for dev environment
βΒ
βββ alembic.ini
βΒ
βββ .env # all the environment variables for the project
βΒ
βββ requirements.txt # only the required dependecies for the application
βΒ
βββ readme.md # documentation for the project
Choice of IDE, Docker, Python3.10, Postgres | PgAdmin
create a new empty project folder, and navigate to this project directory in your terminal. Clone the repository by running the command:
git clone https://github.com/shivajreddy/concha.git
Create the docker images and respective containers altogether using the docker-compose-dev.yml
file. This project embraces Docker compose, to run the Concha Server on local machine and on the server. To start create images, create containers using those images, and starting running the app, simply run the command:
docker compose -f docker-compose-dev.yml up
in this -f
refers to compose and build docker containers using the give file.
To tear down the contaiers simply run.
docker compose -f docker-compose-dev.yml down
Docker Compose does all the heavy lifting of project setup on your local machine especially setting the dev environment:
a: It First builds the image's using the Dockerfile
which is in the root directory. Then it connects to your local machines port 8000
. In any case you want to run this project at a different port other than 8000
, you can change it in the .env
file since the command in the dev compose file overrides the command in the Dockerfile
b: sync the volumes for live changes, between the local machines project directory and containers project directory
c: Create a postgress container dependancy, which will be initialized using the environment variables provided in env
file
d: And link the server
container of python app to the PSQL container.
The project comes with the first alembic revision already set up, this revision basically creates all the tables with approriate conditions and relations. For any further changes in the Database schema in the future, you can easily modify(or add new) models in the psql_db/models.py
and create a new revision using alembic revision -m "<message related to the revision>"
, and then upgrade the database
SSH to container's(dev-server-1
which is running the application) terminal. (Assuming that concha-dev-server-1 is the name of container, if you have changed it, then use the new containers name instead, which you can find using docker container ls
command)
docker exec -it concha-dev-server-1 bash
now you are in the terminal of the container that is running the python app. so for the very first time the app runs(either local machine or server) we have to start the database migrataions, so that our database is upto date with the modal schema defined in the project. to achieve this run the command:
alembic upgrade head
Since there is already a revision that comes with the project, this command brings the database to this revision which in your case is intiating the DB with all the tables with correct schema.
Before running tests, you have to make sure you are using the correct python environment on your local machine. First create virtual environment in the project directory, for example lets call this environment venv
. To create your virtual environment, with your terminal pointing to project's root directory run python -m venv venv
this will create the venv
virtual environment, and now install all the dependencies in the virtual environment, using the 'requirements.txt' file that holds all the necessary dependancies. To do this run: python -m pip install -r requirements.txt
You must create a local database which we can use for testing the application. Create a new database with name concadb_test
pytest
Since 'pytest' is one of the dependancies you installed earlier, this command will start pytest, which will start collecting all the tests, and run them. You can use -s
flag for pytest for displaying print statements in your tests, -v
for verbose(more details), -x
to stop tests if there is any error found.
Postman Collection : Postman
## Running project on GCPGoogle Cloud Platform: GCP
Choose the CPU cores and Memory depending on the amount of traffic that the server must handle.
Go to vpc networks
> Firewall
> CREATE FIREWALL RULE
. Name this rule as allow8000
and choose the network-tag name as allow8000
and for IPv4 ranges write 0.0.0.0/0
, and the checkt TCP
and TCP ports as "8000".
Now go back to VM instances, choose your newly created instance and select 'EDIT' and and the network-tag allow8000
to this instance.
TL;DR GCP by default doesn't allow 8000 port on your VM instances, so you have to create a firewall rule to allow 8000 port to be open to the external IP's. Because FastAPI by default run's the app on port 8000 and we want to keep using that to be consistent.
create a new empty project folder, and navigate to this project directory in your terminal. Clone the repository by running the command:
$ git clone https://github.com/shivajreddy/concha.git
$ cd concha
Run the server using:
docker compose -f docker-compose-prod.yml up -d
This will start the docker containers and run the app in detached mode
SSH to container's(dev-server-1
which is running the application) terminal. (Assuming that concha-dev-server-1 is the name of container, if you have changed it, then use the new containers name instead, which you can find using docker container ls
command)
docker exec -it concha-dev-server-1 bash
now you are in the terminal of the container that is running the python app. so for the very first time the app runs(either local machine or server) we have to start the database migrataions, so that our database is upto date with the modal schema defined in the project. to achieve this run the command:
alembic upgrade head
This project has Github action as part of CI/CD to automatically run all the tests on a linux machine provided by GitHub, by using the GitHub action. The build sequence is written in 'build-deploy.yml' file, which can be found at .github/workflows/build-deploy.yml
in your project source directory.
To make a request, first find the HTTP method and the path(end point) for the operation that you want to use. For example, the "Get all Users" operation uses theΒ GET
Β method and theΒ {base-url}/user/all
Β path.
- Show how to read and use swagger ui, mention how to use /docs
Method | Endpoint | Query params | Form-data | Description |
---|---|---|---|---|
GET | /user/all | No | no | example description alskdfjalksjdflasjdflkasjdfklj |
Search for a user
{{concha}}/user/search
Description: search for a user in the DB using either name or email but not both
Postman Concha
Postman collection for /audio-data end point Postman audio-data collection
Postman collection for /auth end point Postman auth collection
Postman collection for /user end point Postman user collection
-
Incorporate declarative containers using Dependency Injector when application goes large scale
-
Logging
-
[ ]