Add tutorials of how to create new environment about dm_control HOT 9 CLOSED

@huiwenzhang @DanielTakeshi It is really easy, at least for a simple environment:

Create files suite/new_domain.xml and suite/new_domain.py. The .xml file should contain your MuJoCo model in the MJCF format. Have a look at rl/control.py to see what your domain class in suite/new_domain.py must implement, note that some functions are already implemented in suite/base.py. You can also look at some other simple environments (e.g. cartpoly.py and humanoid.py) to get a feeling. You can find a minimalistic example of a suite/robot.py file for a 'robot' domain with a 'test' task below.

"""Robot Domain."""

import collections

from dm_control import mujoco
from dm_control.rl import control
from dm_control.suite import base
from dm_control.suite import common
from dm_control.utils import containers

_DEFAULT_TIME_LIMIT = 30
_CONTROL_TIMESTEP = .04

SUITE = containers.TaggedTasks()

def get_model_and_assets():
  """Returns a tuple containing the model XML string and a dict of assets."""
  return common.read_model('robot.xml'), common.ASSETS

@SUITE.add('playing')
def test(time_limit=_DEFAULT_TIME_LIMIT, random=None, environment_kwargs=None):
  """Returns the Test task."""
  physics = Physics.from_xml_string(*get_model_and_assets())
  task = Robot()
  environment_kwargs = environment_kwargs or {}
  return control.Environment(
      physics, task, time_limit=time_limit, control_timestep=_CONTROL_TIMESTEP,
      **environment_kwargs)

class Physics(mujoco.Physics):
  """Physics class"""

class Robot(base.Task):
  def __init__(self, random=None):
    """Initializes an instance of `Robot`."""
    super(Robot, self).__init__(random=random)

  def initialize_episode(self, physics):
    """Sets the state of the environment at the start of each episode."""
    pass
  
  def get_observation(self, physics):
    """Returns either the pure state or a set of egocentric features."""
    obs = collections.OrderedDict()
    return obs

  def get_reward(self, physics):
    """Returns a reward to the agent."""
    return 0

Once you have implemented your .py file, you just need to import it near the top of suite/_init_.py, and you should be able to use your environment!

For composed environments (e.g. Soccer), it is a bit more complicated, but you should be able to understand it from studying the code.

from dm_control.

Totally agree. Add documents for the API and instructions or examples to create customized environment. Hope it will come soon.

from dm_control.

Just wondering, was there any progress made on this?

from dm_control.

(For future readers): also see some of the comments on #53.

from dm_control.

We have an example of defining a new task from scratch in the Composer section of the tutorial colab

from dm_control.

Dear All,
It is requested that kindly help or provide some link how to design states, Actions and Rewards to train the agent on offline data. Data available on CSV file.

from dm_control.

Please read the the tutorial (Section 6) of the tech report associated with this package and/or run the associated tutorial colab.

from dm_control.

Hi Yuv,

Thanks for writing. I am looking for some py file and notebook on how to design a custom environment to train the agent in RL. I am trying to put some offline data and my actions are in discreet in nature.

Regards,

from dm_control.

Please read the the tutorial (Section 6) of the tech report associated with this package and/or run the associated tutorial colab.

The tutorial file doesn't open in colab or on github the error displayed is "Invalid notebook, The Notebook Does Not Appear to Be Valid JSON"

from dm_control.