Deep Mimic example about deepworlds HOT 10 OPEN

Here is an update on the ghost robot which the robot will need to follow:
This will be used to train the RL algorithm in accordance with Deep Mimic's policies (at least that's the hope for now)

• Some of the data is a bit choppy as it is using IK to follow mocap from the humanoid (which is hidden).
• The speed of motion of each joint, the robot base, and the joint calibration offsets are variable(needs tweaking, some of them are shown).
• There are issues with joint retargeting as of now(right foot of the robot), but, it will be tweaked later.

Warm Regards,
Rohit Kumar J

from deepworlds.

@tsampazk, Thank you for adding me as a contributor!

I am currently working on the Deep Mimic Example in pybullet. Testing methods that would help to parent the stock humanoid so any other similarly structured robot can be used for training without much initial setup. Here are some results:

In this video, the stock humanoid is driving the custom-designed robot using inverse kinematics (I'm hoping that this will be the reward function for the robot during training):

Once, complete, hopefully, we can port this example to Webots.

Warm Regards,
Rohit

from deepworlds.

We are at preliminary training by using IK. The code structure that was initially built upon was... unelegant, however, the agents are stand-alone, hence we may be able to re-build the environment files while reusing the agents(or perhaps use the agents in deep worlds:thinking: :thought_balloon: ). However, this may take some time.

The checkpoint file(agent) here, is at 18 million samples, according to the deep mimic paper, it takes about 61 mil samples for the stock humanoid to achieve a perfect walking gait and 48 mil samples for Atlas. They also mention that it takes 2 days to train the humanoid. To get to the 18 mil samples mark as seen in this video, it took me 24 hours of training with 12(or was it 6? 🤔 ) cores (actually on my friend's PC). I think it needs some tuning to optimize the results.

Hopefully, I could begin developing this example on Webots once this is fully trained.

Warm Regards,
Rohit Kumar J

PS: The sudden jump at 00:12 from the robot was me dragging the robot with the mouse :D

from deepworlds.

@all-contributors please add @rohit-kumar-j for ideas

from deepworlds.

@tsampazk

I've put up a pull request to add @rohit-kumar-j! 🎉

from deepworlds.

@all-contributors please add @rohit-kumar-j for ideas

from deepworlds.

@tsampazk

I've put up a pull request to add @rohit-kumar-j! 🎉

from deepworlds.

@rohit-kumar-j This looks really promising Rohit! I'm looking forward to seeing the complete example, so as to start working on porting it to Webots. I think it would make for an impressive example to be added in the deepworlds repository.

from deepworlds.

@tsampazk, I agree. Unfortunately, I do not know Webots code-base and methods, hence, I can help out with the logic and implementation while simultaneously learning Webots. I hope it is okay if I post updates on the example in this thread itself.

Warm Regards,
Rohit Kumar J

from deepworlds.

I hope it is okay if I post updates on the example in this thread itself.

@rohit-kumar-j Yeap sounds fine, go ahead. 😀

from deepworlds.