elharirymatteo / rans Goto Github PK

Follow the Isaac Sim documentation to install the latest Isaac Sim release.

Examples in this repository rely on features from the most recent Isaac Sim release. Please make sure to update any existing Isaac Sim build to the latest release version, 2023.1.1, to ensure examples work as expected.

git clone https://github.com/elharirymatteo/RANS.git

For Linux: alias PYTHON_PATH=~/.local/share/ov/pkg/isaac_sim-*/python.sh
For Windows: doskey PYTHON_PATH=C:\Users\user\AppData\Local\ov\pkg\isaac_sim-*\python.bat $*
For IsaacSim Docker: alias PYTHON_PATH=/isaac-sim/python.sh

PYTHON_PATH -m pip install -e .

git clone https://github.com/AntoineRichard/rl_games

cd rl_games
PYTHON_PATH -m pip install --upgrade pip
PYTHON_PATH -m pip install -e .

To train your first policy, (example for the USV robot) run:

PYTHON_PATH scripts/rlgames_train_RANS.py task=ASV/GoToPose train=RANS/PPOcontinuous_MLP headless=True num_envs=1024

Modify num_envs appropriately to scale with your current machine capabilities. Turn headless to False if you want to visualize the envs while training occurs.

You should see an Isaac Sim window pop up. Once Isaac Sim initialization completes, the scene for the selected robot will be constructed and simulation will start running automatically. The process will terminate once training finishes.

Here's another example - GoToPose for the Satellite robot (MFP - modular floating platform) - using the multi-threaded training script:

PYTHON_PATH scripts/rlgames_train_RANS.py task=MFP2D/GoToPose train=RANS/PPOmulti_discrete_MLP

Note that by default, we show a Viewport window with rendering, which slows down training. You can choose to close the Viewport window during training for better performance. The Viewport window can be re-enabled by selecting Window > Viewport from the top menu bar.

To achieve maximum performance, launch training in headless mode as follows:

PYTHON_PATH scripts/rlgames_train_RANS.py task=MFP2D/GoToPose train=PPOmulti_discrete_MLP headless=True

Some of the examples could take a few minutes to load because the startup time scales based on the number of environments. The startup time will continually be optimized in future releases.

Checkpoints are saved in the folder runs/EXPERIMENT_NAME/nn where EXPERIMENT_NAME defaults to the task name, but can also be overridden via the experiment argument.

To load a trained checkpoint and continue training, use the checkpoint argument:

PYTHON_PATH scripts/rlgames_train_RANS.py task=MFP2D/GoToPose train=RANS/PPOmulti_discrete_MLP checkpoint=runs/MFP2D_GoToPose/nn/MFP2D_GoToPose.pth

To load a trained checkpoint and only perform inference (no training), pass test=True as an argument, along with the checkpoint name. To avoid rendering overhead, you may also want to run with fewer environments using num_envs=64:

PYTHON_PATH scripts/rlgames_train_RANS.py task=MFP2D/GoToPose train=RANS/PPOmulti_discrete_MLP checkpoint=runs/MFP2D_GoToPose/nn/MFP2D_GoToPose.pth test=True num_envs=64

Note that if there are special characters such as [ or = in the checkpoint names, you will need to escape them and put quotes around the string. For example, checkpoint="runs/Ant/nn/last_Antep\=501rew\[5981.31\].pth"

PYTHON_PATH scripts/random_policy.py task=MFP2D/GoToPose

This script will sample random actions from the action space and apply these actions to your task without running any RL policies. Simulation should start automatically after launching the script, and will run indefinitely until terminated.

PYTHON_PATH scripts/rlgames_train_RANS.py task=MFP2D/GoToPosition

This script creates an instance of the PPO runner in rl_games and automatically launches training and simulation. Once training completes (the total number of iterations have been reached), the script will exit. If running inference with test=True checkpoint=<path/to/checkpoint>, the script will run indefinitely until terminated. Note that this script will have limitations on interaction with the UI.

We use Hydra to manage the config.

Common arguments for the training scripts are:

• task=TASK - Selects which task to use. Examples include MFP2D/GoToPosition, MFP2D/GoToPose, MFP2D/TrackLinearVelocity, MFP2D/TrackLinearAngularVelocity, MFP3D/GoToPosition, MFP3D/GoToPose. These correspond to the config for each environment in the folder omniisaacgymenvs/cfg/task/ + MFP2D or any robot description (eg. ASV for the boat or AGV for the turtle-bots).
• train=TRAIN - Selects which training config to use. Will automatically default to the correct config for the environment file inside the train/RANS folder (e.g., PPOcontinuous_MLP or PPOmulti_discrete_MLP).
• num_envs=NUM_ENVS - Selects the number of environments to use (overriding the default number of environments set in the task config).
• seed=SEED - Sets a seed value for randomization, overriding the default seed in the task config.
• pipeline=PIPELINE - Which API pipeline to use. Defaults to gpu, can also be set to cpu. When using the gpu pipeline, all data stays on the GPU. When using the cpu pipeline, simulation can run on either CPU or GPU, depending on the sim_device setting, but a copy of the data is always made on the CPU at every step.
• sim_device=SIM_DEVICE - Device used for physics simulation. Set to gpu (default) to use GPU and to cpu for CPU.
• device_id=DEVICE_ID - Device ID for GPU to use for simulation and task. Defaults to 0. This parameter will only be used if simulation runs on GPU.
• rl_device=RL_DEVICE - Which device / ID to use for the RL algorithm. Defaults to cuda:0, and follows PyTorch-like device syntax.
• test=TEST - If set to True, only runs inference on the policy and does not do any training.
• checkpoint=CHECKPOINT_PATH - Path to the checkpoint to load for training or testing.
• headless=HEADLESS - Whether to run in headless mode.
• experiment=EXPERIMENT - Sets the name of the experiment.
• max_iterations=MAX_ITERATIONS - Sets how many iterations to run for. Reasonable defaults are provided for the provided environments.
• warp=WARP - If set to True, launch the task implemented with Warp backend (Note: not all tasks have a Warp implementation).
• kit_app=KIT_APP - Specifies the absolute path to the kit app file to be used.

Hydra also allows setting variables inside config files directly as command line arguments. For example, to set the minibatch size for an rl_games training run, you can use train.params.config.minibatch_size=64. Similarly, variables in task configs can also be set, such as task.env.episodeLength=100.

Default values for each of these are found in the omniisaacgymenvs/cfg/config.yaml file.

The way that the task and train portions of the config works are through the use of config groups. You can learn more about how these work here The actual configs for task are in omniisaacgymenvs/cfg/task/<TASK>.yaml and for train in omniisaacgymenvs/cfg/train/<TASK>PPO.yaml.

In some places in the config you will find other variables referenced (for example, num_actors: ${....task.env.numEnvs}). Each . represents going one level up in the config hierarchy. This is documented fully here.