Giter Club home page Giter Club logo

rituai-data-polling's Introduction

rituai-data-ingestion

A short description of the project

Development Requirements

  • Python3.11.0
  • Pip
  • Poetry (Python Package Manager)

M.L Model Environment

MODEL_PATH=./ml/model/
MODEL_NAME=model.pkl

Update /predict

To update your machine learning model, add your load and method change here at predictor.py

Installation

poe install-dev

Runnning Localhost

poe run

Access Swagger Documentation

http://localhost:8080/docs

Access Redocs Documentation

http://localhost:8080/redoc

Running Tests

poe test

Project structure

Files related to application are in the app or tests directories. Application parts are:

app
|
| # Fast-API stuff
├── api                 - web related stuff.
│   └── routes          - web routes.
├── core                - application configuration, startup events, logging.
├── models              - pydantic models for this application.
├── services            - logic that is not just crud related.
├── main-aws-lambda.py  - [Optional] FastAPI application for AWS Lambda creation and configuration.
└── main.py             - FastAPI application creation and configuration.
|
| # ML stuff
├── data             - where you persist data locally
│   ├── interim      - intermediate data that has been transformed.
│   ├── processed    - the final, canonical data sets for modeling.
│   └── raw          - the original, immutable data dump.
│
├── notebooks        - Jupyter notebooks. Naming convention is a number (for ordering),
|
├── ml               - modelling source code for use in this project.
│   ├── __init__.py  - makes ml a Python module
│   ├── pipeline.py  - scripts to orchestrate the whole pipeline
│   │
│   ├── data         - scripts to download or generate data
│   │   └── make_dataset.py
│   │
│   ├── features     - scripts to turn raw data into features for modeling
│   │   └── build_features.py
│   │
│   └── model        - scripts to train models and make predictions
│       ├── predict_model.py
│       └── train_model.py
│
└── tests            - pytest

GCP

Deploying inference service to Cloud Run

Authenticate

  1. Install gcloud cli
  2. gcloud auth login
  3. gcloud config set project <PROJECT_ID>

Enable APIs

  1. Cloud Run API
  2. Cloud Build API
  3. IAM API

Deploy to Cloud Run

  1. Run gcp-deploy.sh

Clean up

  1. Delete Cloud Run
  2. Delete Docker image in GCR

AWS

Deploying inference service to AWS Lambda

Authenticate

  1. Install awscli and sam-cli
  2. aws configure

Deploy to Lambda

  1. Run sam build
  2. Run `sam deploy --guiChange this portion for other types of models

Add the correct type hinting when completed

aws cloudformation delete-stack --stack-name <STACK_NAME_ON_CREATION>

Made by https://github.com/arthurhenrique/cookiecutter-fastapi/graphs/contributors with ❤️

rituai-data-polling's People

Contributors

cwang avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.