facebookresearch / recipes Goto Github PK

Recipes are a standard, well supported set of blueprints for machine learning engineers to rapidly train models using the latest research techniques without significant engineering overhead.Specifically, recipes aims to provide- Consistent access to pre-trained SOTA models ready for production- Reference implementations for SOTA research reproducibility, and infrastructure to guarantee correctness, efficiency, and interoperability.

License: BSD 3-Clause "New" or "Revised" License

Python 99.40% Dockerfile 0.19% Shell 0.41%

recipes's Introduction

TorchRecipes

Train machine learning models with a couple of lines of code with `torchrecipes`.

This library is currently under heavy development - if you have suggestions on the API or use-cases you'd like to be covered, please open an GitHub issue or reach out. We'd love to hear about how you're using torchrecipes!

A recipe is a ready-to-run application that trains a deep learning model by combining a model architecture, trainer, config, etc that you can easily run, modify, or extend. Recipes run on everything from local development environments on your laptop, to large scale clusters. They enable quick experimentation through configuration, and a good starting place to extend the code by forking when more extensive changes are needed.

We provide a number of out-of-the-box recipes across popular domains (vision, NLP, etc) and tasks (image, text classification, etc) that you can use immediately or as a starting point for further work.

Why `torchrecipes`?

Getting started with training machine learning models is a lot easier if you can start with something that already runs, instead of having to write all the glue code yourself.

Machine learning, whether for research or production training, requires working with a number of components like training loops or frameworks, configuration/hyper-parameter parsing, model architectures, data loading, etc. Recipes provide production-ready examples for common tasks that can be easily modified. A recipe at a high-level integrates these modular components so that you can modify the ones that matter for your problem!

We focus our recipes on providing consistent, high-quality baselines that accurately reproduce research papers.

Get Started

Installation

We recommend Anaconda as Python package management system. Please refer to pytorch.org for the detail of PyTorch (torch) installation.

pip install torchrecipes

To install torchrecipes from source, please run the following commands:

git clone https://github.com/facebookresearch/recipes.git && cd recipes
pip install -e .

Vision

Image Classification Recipe

Text

Text Classification Recipe

Anatomy of a Recipe

A recipe is a Python application that you can run directly or customize:

main.py: the entrypoint to start training. The script name doesn't matter and might be different in various recipes.
conf/: Hydra configuration for the training job (including defaults)
module/: the model implementation for pytorch-lightning based recipes
data_module/: the data loading/processing implementation for pytorch-lightning based recipes
tests/: tests for recipe

By default each recipe supports changing common things via configuration files like hyper-parameters as well as changing the model that is loaded itself (e.g. change from ResNet18 to ResNet52). For research and experimentation, you can also override any of the functionality directly by modifying the model, training loop, etc.

Contributing

We welcome PRs! See the CONTRIBUTING file.

License

TorchRecipes is BSD licensed, as found in the LICENSE file.

recipes's People

Contributors

Stargazers

Watchers

recipes's Issues

Failure of installing from source code using Python 3.7

I'm using Python 3.7.13, I checked out master branch and tried to install from source, but it failed with the below error.

[root@~/recipes #]pip3 install -e .
Obtaining file:///root/recipes
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3.7 /usr/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpawqnwy25
       cwd: /root/recipes
  Complete output (17 lines):
  Traceback (most recent call last):
    File "/usr/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
      main()
    File "/usr/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/usr/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 114, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/tmp/pip-build-env-j_o4gy55/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 178, in get_requires_for_build_wheel
      config_settings, requirements=['wheel'])
    File "/tmp/pip-build-env-j_o4gy55/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 159, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-j_o4gy55/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 174, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 27
      if version := os.getenv("BUILD_VERSION"):
                  ^
  SyntaxError: invalid syntax
  ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3.7 /usr/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpawqnwy25 Check the logs for full command output.

How to install ai_codesign?

I'm running torchrec lightning recipe on Ray cluster by following https://github.com/facebookresearch/recipes/tree/main/torchrecipes/rec.
Command:
torchx run -s ray -cfg working_dir=.,dashboard_address=localhost:31024 dist.ddp -j 1x4 --gpu 4 --script ./dlrm_main.py

Error:

ray/0 (CommandActor pid=2748) [2]:Traceback (most recent call last):
ray/0 (CommandActor pid=2748) [2]:  File "./dlrm_main.py", line 25, in <module>
ray/0 (CommandActor pid=2748) [2]:    from torchrecipes.rec.modules.lightning_dlrm import LightningDLRM
ray/0 (CommandActor pid=2748) [2]:  File "/root/recipes/torchrecipes/rec/modules/lightning_dlrm.py", line 18, in <module>
ray/0 (CommandActor pid=2748) [2]:    from ai_codesign.benchmarks.dlrm.torchrec_dlrm.modules.dlrm_train import DLRMTrain
ray/0 (CommandActor pid=2748) [2]:ModuleNotFoundError: No module named 'ai_codesign'
ray/0 (CommandActor pid=2748) [3]:Traceback (most recent call last):
ray/0 (CommandActor pid=2748) [3]:  File "./dlrm_main.py", line 25, in <module>
ray/0 (CommandActor pid=2748) [3]:    from torchrecipes.rec.modules.lightning_dlrm import LightningDLRM
ray/0 (CommandActor pid=2748) [3]:  File "/root/recipes/torchrecipes/rec/modules/lightning_dlrm.py", line 18, in <module>
ray/0 (CommandActor pid=2748) [3]:    from ai_codesign.benchmarks.dlrm.torchrec_dlrm.modules.dlrm_train import DLRMTrain
ray/0 (CommandActor pid=2748) [3]:ModuleNotFoundError: No module named 'ai_codesign'
ray/0 (CommandActor pid=2748) [1]:Traceback (most recent call last):
ray/0 (CommandActor pid=2748) [1]:  File "./dlrm_main.py", line 25, in <module>
ray/0 (CommandActor pid=2748) [1]:    from torchrecipes.rec.modules.lightning_dlrm import LightningDLRM
ray/0 (CommandActor pid=2748) [1]:  File "/root/recipes/torchrecipes/rec/modules/lightning_dlrm.py", line 18, in <module>
ray/0 (CommandActor pid=2748) [1]:    from ai_codesign.benchmarks.dlrm.torchrec_dlrm.modules.dlrm_train import DLRMTrain
ray/0 (CommandActor pid=2748) [1]:ModuleNotFoundError: No module named 'ai_codesign'
ray/0 (CommandActor pid=2748) [0]:Traceback (most recent call last):
ray/0 (CommandActor pid=2748) [0]:  File "./dlrm_main.py", line 25, in <module>
ray/0 (CommandActor pid=2748) [0]:    from torchrecipes.rec.modules.lightning_dlrm import LightningDLRM
ray/0 (CommandActor pid=2748) [0]:  File "/root/recipes/torchrecipes/rec/modules/lightning_dlrm.py", line 18, in <module>
ray/0 (CommandActor pid=2748) [0]:    from ai_codesign.benchmarks.dlrm.torchrec_dlrm.modules.dlrm_train import DLRMTrain
ray/0 (CommandActor pid=2748) [0]:ModuleNotFoundError: No module named 'ai_codesign'

Env:
Python 3.7
Torchrec: 0.1.1
Torch: 1.11.0 + cu113
TorchX: 0.2.0.dev0
Lightning: 1.6.3

I was trying to install ai_codesign from pip but it didn't work out for me.

[root@~/recipes/torchrecipes/rec #]pip3 install ai-codesign
ERROR: Could not find a version that satisfies the requirement ai-codesign (from versions: none)
ERROR: No matching distribution found for ai-codesign
WARNING: You are using pip version 20.2.2; however, version 22.1.2 is available.
You should consider upgrading via the '/usr/bin/python3.7 -m pip install --upgrade pip' command.
[root@~/recipes/torchrecipes/rec #]pip3 install ai_codesign
ERROR: Could not find a version that satisfies the requirement ai_codesign (from versions: none)
ERROR: No matching distribution found for ai_codesign
WARNING: You are using pip version 20.2.2; however, version 22.1.2 is available.
You should consider upgrading via the '/usr/bin/python3.7 -m pip install --upgrade pip' command.

How can I install this module?

cc @colin2328

Leveraging hydra-zen for auto-generated and validated Hydra configs

Hello! Thanks for creating this repo -- it looks like it will be very useful. Hydra + PyTorch Lightning is definitely a killer one-two punch!

I wanted to bring your attention to hydra-zen, which is a library that adds functionality to Hydra and simplifies the process of writing configs. I think that this could be very handy for recipes.

Rather than hand-write YAML configs, you can auto-generate configs for objects using hydra_zen.builds. Furthermore, these config-generating functions provide strict runtime and static checking of the configs, making it trivial to validate all of recipes configs during nightly builds.

Another perk of leveraging hydra-zen is that it can make recipes less dependent on the specific directory layout of your directory structure; e.g. you could avoid having users be required to add a file specifically to torchrecipes/launcher/.

I could go on and on, but I'll leave it at that. I would be happy to provide more details, answer questions, etc. I hope that you find this to be useful 😄

Here are our docs: https://mit-ll-responsible-ai.github.io/hydra-zen/
Our code (our project is very well-tested!): https://github.com/mit-ll-responsible-ai/hydra-zen
And a quick example of using hydra-zen + PyTorch lightning: https://mit-ll-responsible-ai.github.io/hydra-zen/how_to/pytorch_lightning.html