notmahi / minibet Goto Github PK

First of all, this is amazing work !, I was trying to use this to experiment and train a behavior clone model (unconditional) on say a modern rpg/ multiplayer game like pubg/dota/nfs. I was planning to record sequence of game play with image sequence and action sequences.
I am currently not understanding one thing - how do I feed a 2D image into this framework? say my image dimension is 224x224x3, but here in the usage I just see obs_dim = 50, which kind of baffles me as in the demo you have the kitchen environment which should use 2D images? , can you please elaborate how to use this for images ?

Secondly, can you please point where is the loss function being back propagated in the custom training case? Or do we need to define training loop ourselves? does these lines below also includes the loss backprop ? (in the example usage section)

if i<=7:
train_action, train_loss, train_loss_dict = cbet(obs_seq, goal_seq, action_seq)

Thanks