Impact sound when robot colliding with objects? about tdw HOT 7 CLOSED

@xf-zhao See this pull request

This adds impact sounds for robot joints. Please run the test and let me know if it works for you.

In the near future, we plan to simplify the code in the example controller to make it easier to enable impact sounds.

And another question is can we implement (or alternatively) friction sound in TDW?

We're working on this, but it hasn't been implemented yet.

from tdw.

@xf-zhao To follow on from what @alters-mit said above....
We are working on implementing both object scraping and rolling synthesis, to complement the existing impact sound synthesis. These new models will be included in an upcoming release, in the near future.

from tdw.

Thank you very much for the update. I checked your example code, it works fine. Still a minor issue remains for my personal use:

If I set collision_types to ["obj", "env"],

            {
                "$type": "send_collisions",
                "enter": True,
                "stay": False,
                "exit": True,
                "collision_types": ["obj", "env"],
            },

the following error occured when a robot gripper contacts the env (like floor):

  File "main.py", line 187, in wait_execution
    audio_commands = p.get_audio_commands(resp=resp, floor=floor, wall=wall)
  File "~/Cloud/Codes/tdw/Python/tdw/py_impact.py", line 464, in get_audio_commands
    speeds[target] < 0.01 or target in colliders or target in stays_and_exits:
KeyError: -1794863089

1. I checked that in my case the key -1794863089 is the joint_id of the gripper_link, and it crashes when the gripper touches the floor.
2. If I set collision_types to ["obj"], everything works fine with no errors.
3. Anyway, I can avoid such error by limit the gripper's y position to greater than 0, i.e. above floor.

BTW, the impact sound effect looks overall great. But I am still curious about if there any methods to control the threshold of impact sound? (to something like even minor contact will produce sound instead of being quite, maybe there are already small waves if I recorded it to a .wav file but I cannot hear on the fly?).

from tdw.

Anyway, I can avoid such error by limit the gripper's y position to greater than 0, i.e. above floor.

This is a known issue and the upcoming TDW 1.9.0 release will make it easier to manage. Basically, the build crashes if collision detection is enable while the end effector is slightly intersecting with the floor. The update will make it much easier to move the robot to an "initial position" before enabling collision detection.

But I am still curious about if there any methods to control the threshold of impact sound?

PyImpact has an initial_amp parameter that should help with this. Typically, I just use a frame counter-- if there is sound for over n frames, I assume that it's really quiet and stop recording.

from tdw.

Also, there is an amp parameter set for every audio-enabled object, in the objects.csv spreadsheet. You can always increase the amp value for a specific object to control the amplitude of the impact sound generated for that object.

from tdw.

This is a known issue and the upcoming TDW 1.9.0 release will make it easier to manage.

Thank you again for your help. Hope to see the new version soon. :)

from tdw.

@xf-zhao

We just released TDW v1.9.0

This document explains the new robotics API.

This document explains collision detection with robots.

This document explains the new PyImpact API.

This is the v1.8 to v1.9 upgrade guide.

from tdw.