Giter Club home page Giter Club logo

mlopsproject's Introduction

MLOpsProject

A project for the Machine Learning Operations course based around a Super Resolution model.

How To Run

Use make help to see how to run important features with descriptions.

Project Organization

├── LICENSE
├── Makefile                <- Makefile with commands like `make data` or `make train`
├── README.md               <- The top-level README for developers using this project.
├── azure                   <- Contains scipts for deploying/training models using Microsoft Azure.
├── data
│   ├── external            <- Data from third party sources.
│   ├── interim             <- Intermediate data that has been transformed.
│   ├── processed           <- The final, canonical data sets for modeling.
│   └── raw                 <- The original, immutable data dump.
│
├── docs                    <- A default Sphinx project; see sphinx-doc.org for details. CURRENTLY NOT IN USE.
│
├── models                  <- Trained and serialized models, model predictions, or model summaries.
│
├── notebooks               <- Jupyter notebooks. Naming convention is a number (for ordering),
│                              the creator's initials, and a short `-` delimited description, e.g.
│                              `1.0-jqp-initial-data-exploration`.
│
├── references              <- Data dictionaries, manuals, and all other explanatory materials.
│
├── reports                 <- Generated analysis as HTML, PDF, LaTeX, etc.
│   └── figures             <- Generated graphics and figures to be used in reporting.
│
├── requirements.txt        <- The requirements file for reproducing the analysis environment, e.g.
│                              generated with `pip freeze > requirements.txt`.
├── requirements_test.txt   <- The requirements file for running the tests.
├── setup.py                <- Makes project pip installable (pip install -e .) so src can be imported
├── src                     <- Source code for use in this project.
│   ├── __init__.py         <- Makes src a Python module
│   │
│   ├── data                <- Scripts to download or generate data
│   ├── hparams             <- .yaml files for hyperparameter configuration using Hydra.
│   ├── models              <- Scripts to train models and then use trained models to make
│                              predictions
│── tests                   <- Test scripts using pytest.

Project Checklist

The following checklist gives a good sense of what is included in the project:

Week 1

  • Create a git repository
  • All members have write access to repository
  • Using dedicated environment to keep track of packages
  • File structure made using cookiecutter
  • make_dataset.py filled to download needed data
  • Add a model file and a training script and get that running
  • Done profiling and optimized code
  • requirements.txt filled with used dependencies
  • Write unit tests for some part of the codebase and get code coverage
  • Get some continues integration running on the github repository
  • use either tensorboard or wandb to log training progress and other important metrics/artifacts in your code
  • remember to comply with good coding practices while doing the project

Week 2

  • Setup and used Azure to train your model
  • Played around with distributed data loading
  • (not curriculum) Reformated your code in the pytorch lightning format
  • Deployed your model using Azure
  • Checked how robust your model is towards data drifting
  • Deployed your model locally using TorchServe

Week 3

  • Used Optuna to run hyperparameter optimization on your model
  • Wrote one or multiple configurations files for your experiments
  • Used Hydra to load the configurations and manage your hyperparameters

Additional

  • Revisit your initial project description. Did the project turn out as you wanted?
  • Make sure all group members have a understanding about all parts of the project
  • Created a powerpoint presentation explaining your project
  • Uploaded all your code to github

Project based on the cookiecutter data science project template. #cookiecutterdatascience

mlopsproject's People

Contributors

philipwastakenwastaken avatar blackshyguy avatar freddymurphy avatar rborstroem avatar actions-user avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.