Image and dict observation spaces support by BC and GAIL implementations about di-engine HOT 1 CLOSED

Hello @akmandor.
As for GAIL: observations space can be both a 1D array or a 3D array and action space is a 1D array. In our implementation, observations space is not a dict, but we have taken this into account and included it in the roadmap. For now it can be a numpy array or a tensor.
For a quick example you can run the following commands:

cd dizoo/box2d/lunarlander/config
ding -m serial -c lunarlander_dqn_config.py   # train lunarlander expert DQN
python lunarlander_dqn_gail_config.py  # collect expert data and train lunarlander GAIL

BC is still under development and you can follow its progress in this PR: #110

Note that we have not tested GAIL with 3D observations yet, but we have extensively tested it on Mujoco and Box2D envs. If you would like to write your own 3D encoder for GAIL, you can replace it with our encoder here: https://github.com/opendilab/DI-engine/blob/main/ding/reward_model/gail_irl_model.py#L75

If you need to write an encoder that supports dict observations, you can modify our encoder in a way similar to this according to the design of your environment:

def forward(self, x) -> torch.Tensor:
	x1, nx = x[‘obs1’], x[‘obs_n’]
        x1, xn = preprocess(x1, xn)
        out = torch.Tensor(np.concat(x1, nx))
        out = self.l1(out)
        out = self.a1(out)
        out = self.l2(out)
        out = self.a2(out)
        return out

from di-engine.