um-arm-lab / unity_victor_teleop Goto Github PK

Unity project for teleoperation of ARM-lab's "Victor" robot using an HTC Vive

License: Apache License 2.0

C# 98.11% ShaderLab 0.47% HLSL 0.07% CMake 0.03% Objective-C 0.41% SCSS 0.92%

unity_victor_teleop's Introduction

Unity Teleop (for ARM lab's "Victor" robot)

This is the unity portion use for teleoperating a robot. Want to learn how to use this package to teleop your robot? Check out the tutorials!

Brief description

This is a Unity project for controlling the armlab's Victor robot using an HTC vive. This unity project connects to VR Teleop, connecting the Unity interface to the ROS ecosystem. Video from a Kinect enter as ROS rgb and depth images. The current robot position enters as ROS robot_state messages The commanded gripper poses exit as geometry_msgs::pose.

Installation

Install unity (Free version)
Clone this repo
Open this repo as a unity project
Import SteamVR from the unity asset store
- Window -> Asset Store and search for SteamVR

(If you are having issues, check out the tutorials)

Ubuntu Setup

To communicate to ROS: Set up an ubuntu computer with ROS, optionally a VM. Follow the instruction provided by ROSSharp [https://github.com/siemens/ros-sharp/wiki/User_Inst_ROSOnUbuntu]

Set up VR teleop on ubuntu: https://github.com/UM-ARM-Lab/vr_teleop

For more information, see the wiki

unity_victor_teleop's People

Contributors

Stargazers

Watchers

Forkers

nusisschad stevensu1838 gijaeahn

unity_victor_teleop's Issues

Laggy motion

Sometimes when using Victor the arms + gripper become laggy.

Repro steps (under some conditions, but not always?):

"Grasp" arm with VR controller
Close/open gripper. Responds as normal
Hold the gripper for awhile, maybe move around the arms
close/open gripper: there is significant lag.

The problem is worse when you use both arms at the same time

Reduce or eliminate buffer of kinect data

Currently the image/depth_image subscriber has a buffer of images not yet processed. If the kinect data is published faster than it can be processed in unity, this creates a large delay between real world motions and the point cloud in VR.

This buffer should be removed so the latest kinect image is always processed, older images are forgotten.

Unity screen gray's out occasionally

The view from the Vive occasionally turns completely gray, which likely means the headset has lost tracking, i.e. the lighthouses no longer "see" the headset.

Display Video from a Kinect enter as ROS rgb and depth images in Unity and nothing more

Hi,

I've got a same 3D Kinect camera and HTC vive as you do. And I simply want to generate the realtime point could in Unity 3D to immerse myself in the remote scene as you do in the pancake video. No robot involved now. Could you please tell me how I can achieve this first step?

Which of your project is ready to run in unity? Maybe I can get inspired by running you project.
I did try to run your unity_victor_teleop in Unity to see how you display the image date in Unity but heaps of errors.

Could you please make me a few screen shots of your unity project showing me how exactly the data is processedd and displayed in unity coming through image listener and depth image listener?

I am new but very intrested in your project. Could you please kindly show me how to achieve this? You are the legend

Thanks a million. Cheers.

Increase robustness to installation on new computers

Others have reported unity is unable to find some files when installing this project. This issue can be resolved by testing install and fixing any issues on other computer

Hand physics should be turned off

The hand physics causes the hands collide with the robot. While this looks "cool", it is both annoying because it prevents the user from lining up their own hands with Victor's and causes excess computation.

The fix is simple: Uncheck the Hand Physics script in Unity under (Player -> SteamVRObjects -> LeftHand/RightHand).
I will wait to fix this until we get back into lab and can test the full system again on the real robot

Make Standalone Kinect Unity Demo

Many people have a RGBD sensor, but only our lab has a "Victor" Robot.
I've received several requests for explanations of the kinect data specifically. This could be made into a demo without Victor involved at all, which would make this easier to install on other systems

Make Standalone Kinect Unity Demo

This is a clone of #2 however that thread turned into a separate conversation.

Many people have asked for a unity demo showing how to view kinect data. I can make a standalone demo package just for that, without the complexities of a robot.

License ?

Hello, under what license is this project released under ? Thank-you.