TIPO 1 Download TIPO 2 Primary link
Oculus Integration (v13) now supports hand tracking (via link) directly in editor.
Currently Oculus Link -> Unity integration is not supporting hand tracking. This makes quick iteration for hands related interactions more difficult.
You can use this package to bridge that gap till Link supports that completely. It transmits hand-related data directly to Unity via network and then feeds that into the scripts.
You can see hands in the Scene View and access hand-related data quickly. That should make iterations faster without the need to deploy all changes to Quest.
- Create new scene
- Import Oculus Integration package (you can skip importing SimpleFramework)
- Remove
Main Camera
game object - Add
OVRCameraRig
fromAssets\Oculus\VR\Prefabs
- Make sure
Hand Tracking Support
is set toControllers And Hands
- if you can't see that option make sure platform in build settings is set to
Android
- Make sure
- Add
OVRHandPrefab
and rename toLeftHand
- in
OVRSkeleton
script setUpdate Root Pose
andEnable Physics Capsules
- in
- Add
OVRHandPrefab
and rename toRightHand
- in
OVRSkeleton
script setUpdate Root Pose
andEnable Physics Capsules
- change
Hand Left
toHand Right
inOVR Hand
,OVR Skeleton
amdOVRMesh
- in
- Download and import package
- Go to
/Assets/RemoteHandsTracking/Prefabs
and addOVRHandsDataTransmission
to the scene
- In editor go to
OVRHandsDataTransmission\Feeders
and assignLeftHand
andRightHand
to respectiveOVRHand
game objects for all 3 feeders:
HandsDataFeeder
SkeletonDataFeeder
MeshDataFeeder
- you might need to adjust
OVRHand.GetHandState
method in\Assets\Oculus\VR\Scripts\Util\OVRHand.cs
to
private void GetHandState(OVRPlugin.Step step)
{
if (OVRPlugin.GetHandState(step, (OVRPlugin.Hand)HandType, ref _handState))
{
<omitted>
}
else
{
#if UNITY_EDITOR
//in editor don't change _isInitialized - this could cause feeders adding data at invalid moment (depending on call order) - which will result in no hands being visible
return;
#endif
_isInitialized = false;
}
}
- if your hand are 'flickering' you can also adjust
OVRMeshRenderer.Update
method in\Assets\Oculus\VR\Scripts\Util\OVRMeshRenderer.cs
to
private void Update()
{
if (_isInitialized)
{
bool shouldRender = false;
if (_dataProvider != null)
{
var data = _dataProvider.GetMeshRendererData();
//shouldRender = data.IsDataValid && data.IsDataHighConfidence;
shouldRender = true; //always show hands
}
if (_skinnedMeshRenderer != null && _skinnedMeshRenderer.enabled != shouldRender)
{
_skinnedMeshRenderer.enabled = shouldRender;
}
}
}
- Import Oculus Integration package
- Run
HandsIntegrationTrainScene
from/Assets/Oculus/SampleFramework/Usage
- Make sure you follow Oculus guidelines to set up, also to enable hands in the scene
Hand Tracking Support
onOVRCameraRig
game object needs to be set to eitherHands
orController and hands
. - Download
- Go to
/Assets/RemoteHandsTracking/Prefabs
and addSampleFrameworkHandsDataTransmission
to the scene
- You'll need to add
Hands
game object reference toFeeder
objects
HandsDataFeeder
SkeletonDataFeeder
MeshDataFeeder
- If hands on screen are flickering go to
OculusSampleFramework.Hand.ScaledAlpha
in\Assets\Oculus\SampleFramework\Core\CustomHands\Scripts\Hand.cs
property and set getter to always return '1f'.
// Calculated Alpha vlaue of the hand according to OVRPlugin.TrackingConfidence
public float ScaledAlpha
{
get
{
return 1f; //HACK: always visible
}
set
{
_scaledAlpha = value;
}
}
That value is constantly overridden by default scripts. Even though it's set every there's some timing issue somewhere which could cause hands to flicker.
Data will be sent over your network, best if your PC and Quest are on the same wi-fi network.
- In
HandsDataSender
game object specify IP address that your PC is on- you can get that by running
ipconfig
in console
- you can get that by running
- Make sure your firewall is allowing connections on that IP/port
- In
HandsDataReceiver
specify the same IP and port (do not use loopback address127.0.0.1
, for some people this is causing issues and data will not come through.)
If quest application freezes on start it can not connect to IP Address/port you provided. Please make sure Quest and PC are on the same network, your FW rules are allowing connection and that your router is correctly passing traffic to PC
That's it. Now run the application on Quest and hit play in the editor. Once Quest starts sending your hand data into Unity you'll be able to see them directly in Scene View.
If you're iterating over specific gesture for a while it may be easier to record it and replay, this way you won't need to do it over and over in Quest.
- Play the scene (in editor and in Quest)
- make sure you can see hand movement on-screen before recording
- Go to
HandsDataRecorder
and fillNew Recording Name
- you can have multiple recordings and they'll be locally persisted under that name
- In editor on
HandsDataRecorder
clickStartRecording
- Perform gestures as needed
- In editor on
HandsDataRecorder
clickStopRecording
- you'll now have a child under
HandsDataRecorder
named as defined inNew Recording Name
, this is locally persisted and can be re-run later without needing to even connect Quest to PC
- you'll now have a child under
- Run the scene (you don't need to have Ouest running)
- In editor go to
HandsDataRecorder
- it'll have children with your custom recordings - Pick one that you want to replay and click on
Play
- You can also remove it from drive via
Delete
button
- You can also remove it from drive via
- Inspector will focus on
HandsDataPlayer
and will start playing your pre-recorded gestures
Controlling Replay
With recording assigned to HandDataPlayer
, you can also choose a specific frame that you'd like to view via ManuallyMoveToFrame
You can easily use the package in your project as long as your using OVRHand
prefabs.
If you're using another method to render hands then you'll have to adjust feeder scripts that set hand/mesh/skeleton data. Look at classes in Customisation
to see how that's done. Assign new handler to corresponding event on HandsDataReceiver
object.
Remember to also change event calls on HandDataRecorder
to use your custom feeder objects.
- If your hands are a bit oddly shaped (on screen) make sure your actual hands are not visible to Quest cameras for a second or two when the app starts up. There seems to be some timing/initialization issue with skeleton/mesh if they are visible from very first moments.
- sometimes when replaying mesh data will not be correctly loaded and hands won't render, when that happens just click
Replay
If quest application freezes on start it can not connect to IP Address/port you provided. Please make sure Quest and PC are on the same network, your FW rules are allowing connection and that your router is correctly passing traffic to PC