Giter Club home page Giter Club logo

License: Apache 2 Polyaxon API Slack

Docs Release GitHub GitHub

CLI Haupt Hypertune Traceml Codacy Badge

Reproduce, Automate, Scale your data science

Welcome to Polyaxon, a platform for building, training, and monitoring large scale deep learning applications. We are making a system to solve reproducibility, automation, and scalability for machine learning applications.

Polyaxon deploys into any data center, cloud provider, or can be hosted and managed by Polyaxon, and it supports all the major deep learning frameworks such as Tensorflow, MXNet, Caffe, Torch, etc.

Polyaxon makes it faster, easier, and more efficient to develop deep learning applications by managing workloads with smart container and node management. And it turns GPU servers into shared, self-service resources for your team or organization.


demo


Install

TL;DR;

  • Install CLI

    # Install Polyaxon CLI
    $ pip install -U polyaxon
  • Create a deployment

    # Create a namespace
    $ kubectl create namespace polyaxon
    
    # Add Polyaxon charts repo
    $ helm repo add polyaxon https://charts.polyaxon.com
    
    # Deploy Polyaxon
    $ polyaxon admin deploy -f config.yaml
    
    # Access API
    $ polyaxon port-forward

Please check polyaxon installation guide

Quick start

TL;DR;

  • Start a project

    # Create a project
    $ polyaxon project create --name=quick-start --description='Polyaxon quick start.'
  • Train and track logs & resources

    # Upload code and start experiments
    $ polyaxon run -f experiment.yaml -u -l
  • Dashboard

    # Start Polyaxon dashboard
    $ polyaxon dashboard
    
    Dashboard page will now open in your browser. Continue? [Y/n]: y

compare dashboards


  • Notebook
    # Start Jupyter notebook for your project
    $ polyaxon run --hub notebook

compare


  • Tensorboard
    # Start TensorBoard for a run's output
    $ polyaxon run --hub tensorboard -P uuid=UUID

tensorboard


Please check our quick start guide to start training your first experiment.

Distributed job

Polyaxon supports and simplifies distributed jobs. Depending on the framework you are using, you need to deploy the corresponding operator, adapt your code to enable the distributed training, and update your polyaxonfile.

Here are some examples of using distributed training:

Hyperparameters tuning

Polyaxon has a concept for suggesting hyperparameters and managing their results very similar to Google Vizier called experiment groups. An experiment group in Polyaxon defines a search algorithm, a search space, and a model to train.

Parallel executions

You can run your processing or model training jobs in parallel, Polyaxon provides a mapping abstraction to manage concurrent jobs.

DAGs and workflows

Polyaxon DAGs is a tool that provides container-native engine for running machine learning pipelines. A DAG manages multiple operations with dependencies. Each operation is defined by a component runtime. This means that operations in a DAG can be jobs, services, distributed jobs, parallel executions, or nested DAGs.

Architecture

Polyaxon architecture

Documentation

Check out our documentation to learn more about Polyaxon.

Dashboard

Polyaxon comes with a dashboard that shows the projects and experiments created by you and your team members.

To start the dashboard, just run the following command in your terminal

$ polyaxon dashboard -y

Project status

Polyaxon is stable and it's running in production mode at many startups and Fortune 500 companies.

Contributions

Please follow the contribution guide line: Contribute to Polyaxon.

Research

If you use Polyaxon in your academic research, we would be grateful if you could cite it.

Feel free to contact us, we would love to learn about your project and see how we can support your custom need.

polyaxon's Projects

adlfs icon adlfs

fsspec-compatible Azure Datake and Azure Blob Storage access

charts icon charts

Helm charts for creating reproducible and maintainable deployments of Polyaxon with Kubernetes.

cli icon cli

Polyaxon Core Client & CLI to streamline MLOps

common icon common

Common APIs and libraries shared by other Kubeflow operator repositories.

dash-polyaxon-demo icon dash-polyaxon-demo

This app shows how to explore 3-D chest tomography data using Dash managed by Polyaxon

gcsfs icon gcsfs

Pythonic file-system interface for Google Cloud Storage

haupt icon haupt

Lineage metadata API, artifacts streams, sandbox, API, and spaces for Polyaxon

hub icon hub

Public reusable components for Polyaxon

hypertune icon hypertune

A library for performing hyperparameter optimization

kubernetes icon kubernetes

Production-Grade Container Scheduling and Management

mloperator icon mloperator

Machine Learning Operator & Controller for Kubernetes

mpi-operator icon mpi-operator

Kubernetes Operator for Allreduce-style Distributed Training

polyaxon icon polyaxon

MLOps Tools For Managing & Orchestrating The Machine Learning LifeCycle

polyaxon-lib icon polyaxon-lib

Deep Learning and Reinforcement learning library for TensorFlow for building end to end models and experiments.

polyaxon-nfs-provisioner icon polyaxon-nfs-provisioner

Polyaxon in-cluster NFS provisioner to simplify the creation of ReadWriteMany and ReadOnlyMany volumes.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.