Improvements for dm_control Compatibility Wrapper about shimmy HOT 4 CLOSED

Yes, you are completely correct. Thanks for catching this error, I will fix it now

from shimmy.

I think that @ottofabian wished to add access to the control_timestep with

@property
def dt(self):
    return self.model.opt.timestep * self.frame_skip

When he had time

from shimmy.

@ottofabian Hey, yes these seem like very good suggestions. Could you make a PR that implements both of the suggestions?

from shimmy.

I can do that, but I am currently busy with something else and this will probably take some time.

An another note, is the termination and truncation correctly implemented?
Based on the dm_control code,

   if self._step_count >= self._step_limit:
      discount = 1.0
    else:
      discount = self._task.get_termination(self._physics)

    episode_over = discount is not None

    if episode_over:
      self._reset_next_step = True
      return dm_env.TimeStep(
          dm_env.StepType.LAST, reward, discount, observation)
    else:
      return dm_env.TimeStep(dm_env.StepType.MID, reward, 1.0, observation)

I think it should be reversed, i.e. discount==0 is a termination. For the time limit (aka truncated) the discount is 1.0, 0.0 for failure or final step of finite horizon tasks, such as for LQR, and any other value !=0 when the environment artificially ends the task.
From a consistency perspective, 1.0 also makes more sense given the default value is 1.0 for intermediate and steps.
Some examples and details are also provided in the dm_control documentation.

from shimmy.