Extraction of the demos velocities for imitation learning about rlbench HOT 10 CLOSED

Hi,
You are trying to fetch dems that do not exist. 'Saved' demos will be made available soon.
Instead of:

demos = task.get_demos(2)  # -> List[List[Observation]]

do:

demos = task.get_demos(2, live_demos=True)  # -> List[List[Observation]]

This is similar to what is happening in the imitation learning example.

from rlbench.

I actually did that. I used live demos but as you mentionned in the following line
demos = task.get_demos(2, live_demos=True) # -> List[List[Observation]]
I just had a list of observations only, there are not velocities. In imitation learning I need the velocities for training. How to extract them from the demos because I read your code there are no mention of them?

from rlbench.

I'm having the same issue. Did you solve it?

from rlbench.

The Observation class stores all of the data modalities as instance attributes, so you should be able to just do obs.joint_velocities where obs is one observation from your list of observations. In my data collection pipeline I've been doing something like this:

RLBENCH_MODALITIES = ['joint_positions', 'joint_velocities', 'joint_forces',
                      'gripper_open', 'gripper_pose', 'gripper_joint_positions', 'gripper_touch_forces',
                      'left_shoulder_rgb', 'left_shoulder_depth', 'left_shoulder_mask',
                      'right_shoulder_rgb', 'right_shoulder_depth', 'right_shoulder_mask',
                      'wrist_rgb', 'wrist_depth', 'wrist_mask']
demo, = task_env.get_demos(1, live_demos=True)
demo_data = {[] for modality in RLBENCH_MODALITIES}
for obs in demo:
    for attr in RLBENCH_MODALITIES:
        modality_data = np.expand_dims(getattr(obs, attr), 0)
        demo_data[attr].append(modality_data)

This just builds up a dictionary of the data which I can then easily convert to numpy arrays and save to an HDF5 file.

from rlbench.

Thanks for your input, the problem is that calling obs.joint_velocities returns None

from rlbench.

For Imitation Learning example

from rlbench.

Are all of the other data attributes populated? That is, is the problem specific to only joint velocities, or are there other modalities that are also None?

from rlbench.

Only images are not, everything else is.

from rlbench.

Ok I think I found the issue, in line 22 we are setting everything to false which includes the velocities.

So the solution is to enable them after this line, e.g. obs_config.joint_velocites = True

from rlbench.

Hi,

Yes the example was disabling velocity observations. I'll change it such that all the examples in V1.0.6 will have all observations on :)

from rlbench.