alvr-org / alvr Goto Github PK

I was trying out Vivecraft with your experimental version 5 (which is excellent, by the way) and the controllers didn't work at all. After googling, I found out that ALVR doesn't currently support the SteamVR input system that the current versions of vivecraft use. For your reference: http://www.vivecraft.org/vivecraft-steamvr-input/

If you could add that to the build I'd really appreciate it! Thanks for your great work.

Experimental v5 (ALVR server binaries from GitHub release and ALVRClient built from JackD83/ALVRClient@dbaaf13 or JackD83/ALVRClient@7a20a4e + JackD83/ALVRClient#1) is constantly crashing when starting Steam.

Experimental v3 (ALVR server binaries from GitHub release and ALVRClient built from source with the same modification) is working.
All ALVR settings (%LOCALAPPDATA%\ALVR folder) on PC were deleted. FFR was not enabled.

Excerpt from the crashlog:

Exception: Unhandled Exception.
ExceptionCode=E06D7363
Address=000007FEFD2AB87D (000007FEE4DF0000 + 00000000184BB87D)
OSVer: MajorVersion=6 MinorVersion=1 Build=7601
Module: 000007FEE4DF0000
----------------------------------------
[00:20:18.089 698] got tracking info 3582 -0.313637 -0.078612 -0.065789 0.943994
[00:20:18.089 698] CONTROLLER 0 0.000000,0.000000,0.000000 - 0.000734,-0.004234,0.005565
[00:20:18.089 698] Controller0 0 1: 00000000 00000013 0.000000:0.000000
[00:20:18.089 698] CONTROLLER 1 0.000000,0.000000,0.000000 - -0.000308,0.000604,0.002268
[00:20:18.089 698] Controller1 1 2: 00000000 00000011 0.000000:0.000000
[00:20:18.089 698] Rotation Matrix=(0.978984, 0.173519, -0.107151, 0.000000) (-0.074897, 0.794608, 0.602485, 0.000000) (0.189685, -0.581798, 0.790905, 0.000000)
[00:20:18.089 698] GetPose: Rotation=(-0.313637, -0.078612, -0.065789, 0.943994) Position=(0.010037, -0.055899, -0.135191)
[00:20:18.089 698] Try to send.
[00:20:18.089 698] Poller::Do(). Select 10000 us
[00:20:18.089 698] Cuda kernel execution failed. code=18 invalid texture reference
[00:20:18.089 698] Unhandled Exception.
ExceptionCode=E06D7363
Address=000007FEFD2AB87D (000007FEE4DF0000 + 00000000184BB87D)
[00:20:18.089 698] GetNextSwapTextureSetIndex 0000000040001280 0000000040001340 0 0
========== Exception info start ==========
ExceptionCode=E06D7363 Address=000007FEFD2AB87D ThreadId=7388
RaiseException(RaiseException) +61
========== Exception info end ==========

Has anybody else faced similar issue?

Assuming this uses the Nvidia SDK, than a Linux port should be feasible:
https://www.phoronix.com/scan.php?page=news_item&px=Video-Codec-SDK-9.0-Stable

SteamVR is supported under Linux and with Proton a surprising number of games can be made to run.

I realize that this is a tiny niche target, but with a high percentage of programmers that might be interested in contributing :)

Setting server IP in ALVRClient while it is not running causes a crash. I will fix this.

Workaround is to start ALVRClient and set the server IP while ALVRClient is running. ALVRClient needs to be restarted for the setting to take effect.

As soon as I click on any of the setting ticks the steam sounds settings can't be changed anymore. The sound is stuck on permamently. I have to delete the config file if I want to fix it temporarily.
I installed vb-cable, restarted pc and tried using the mic but I get no sound, all I see in the sound settings under quest mic options is the same message about requiring vb cable. I tried setting vb cable as my default mic. It didn't help. I do have the new versions installed. On top of that I noticed more packet loss than usual. That could be the latest oculus update so that needs more investigating. After a bit of testing I only get packet drops while mic support is enabled even if it isn't working atm.

Hello JackD83!

Thanks to your great jobs.

I have a request.

I have been using ALVR with index controller and some trackers. In Alvr alpha 5, I can disable couclus controller, but Now I can't. So I have to remove battery every time for using ALVR.

Can you add option for disable controller?

Thank you in advanced.

Hi,
I am getting this error and server crashes:
Failed to initialize CEncoder. All VideoEncoder are not available. VCE: AMF Error 1. g_AMFFactory.Init(), NVENC: This GPU does not support H.265 encoding. (NvEncoderCuda NV_ENC_ERR_INVALID_PARAM)
Regardless if I choose H.264 or H.265 - same problem. Please help, thanks.
Windows 10 Pro, 64 bit, Nvidia Geforce 1080 ti.

Video Resolution is reset to 100% when starting ALVR. It was already an issue, but it was fixed back then.

I have a little question that may not be the place to ask but I have no idea how else ask about it (new to GitHub still).

How does ALVR handles resolution on both sides PC and Quest?

I assume that if I set res to 100%, on PC I am rendering the games at the sale resolution of Quest and that's fine.

When I go through tests for latency I have used 50 and even 25% resolutions and I noticed not just less latency but also a performance increment on the games fps itself so I assume that the PC it's rendering also at a 50 or 25% resolution.

But I have also noticed that when I do this, it seems like the Quest internal resolution also changes? Because the little white room on the app on Quest (the waiting to connect room) also changes resolution and that makes me think that the Quest internal resolution it's changing too? Not the screen res itself but maybe the rendering one?

I am wondering this regarding the FFR sceneario.
Let's say that a portion of the image it's 1024px at 100% res then I set res at 50% so now that same portion it's 512px (I am just using numbers as examples, it doesn't matter if they doesn't match :P). If the resolution changes on the video and the Quest rendering it wouldn't matter because they both match. But in the case of using FFR, if I play at 50%, thanks to FFR this same portion would still be 1024px thanks to it but if the internal render on Quest still renders at 50% rather than 100% and stretch the image then we would be rendering that same 1024px portion at only 512 again.

I suposse this is all already though but this is more regarding my curiosity and wanting to learn continuously.

Also, how does exactly FFR works?
Where does the saving happens? On the recording video itself saving encoding time but being sent on the same resolution as if FFR were disabled or it does indeed record at a lower resolution making it not just faster to encode but also less necessary bitrate (like setting it at 75% but looking similar to 100% for example)?

In the case of the video being directly rendered at lower resolution, would we need to apply a different res to take advantage of FFR or this happens automatically? Like, for example, would we need to set it to 75% so we save performance while still looking like 100% or this is automatic and even if we set it at 100% the video will be still encoded at less resolution? Asking because if this is the case it would be cool when we enable FFR to see the equivalent of the final resolution video since there are some settings to adjust there it would be helpful to see how these would affect the final video res (if they do at all).

Of course, if the video is not rendered at a diffrent resolution and it's just save on encoding itself then the last paragraph have no meaning :P

I couldn't try the last build yet so if something of this it's obvious from the new settings there (if any) I am sorry for the redundant questions. Regards♥!

If I may suggest adding a message in ALVR that recommends setting the 5 GHz channel width of your router or access point down from 80 to 40 MHz.

I've been testing and searching for the weird artifact stuttering that happens every few minutes and this is what solves it. Multiple people have been reporting this issue on literally all streaming services: AMD Relive VR, VD, ALVR, VRidge. I think the best solution is to add a warning message in those services. The Oculus Quest's WiFi chip or driver is to blame for this.

Please note that not all routers and access points allow you to adjust this setting. I had no luck adjusting this in my AVM FritzBox nor FritzRepeater. From googling, I think these router manufacturers allow adjusting this setting: Linksys, D Link, TP Link, MikroTik. Netgear and AVM FritzBox don't, as far as I can tell. I haven't searched for any others.

Edit: If anyone is looking for a good access point. I got the MikroTik wAP ac but it was difficult to set up and the connection was terrible. I sent it back and just got a TP Link RE650. Set up is easy and relatively quick. It is working really well. 5 GHz speed at 40 MHz is 400 Mbit (this might actually be the 40 MHz limit of the Quest's WiFi chip).

So I've got this checked, and I use vb-cable frequently so I know how all the audio routing works, but I can't seem to actually get this setting to do anything. Is there any guide explaining how I can use the quest microphone as my mic in game?

Excellent work on this ALVR project. I was playing some pavlov earlier and the quality is just unrivaled. Truly a great experience.

Hello, thanks for your great work.

I cannot get microphone audio to work. I have install virtual cable driver. On ALVR first run in quest I have accepted microphone permissions.

In ALVR windows app I ahve checked "stream quest microphone"

In windows control panel-->manage audio devices I have : Playback Speakers Realtek high definiton audio
and recording: Cable output VB audio virtual cable set as default device and default communication device.

In steamVR settings I have
When using steam VR

Set playback device to (do not change)
Set record device to Cable output

Despite all this microphone seems to not be working. At least no one answers me when I ask them. Is something wrong with my setup and is there any easy way to test my microphone?

Hi, I ve been using Jacks version since first release.
I had no problem with v4 version.
Just updated steamvr and also v5. Now controllers are vive pro looks and menu button keeps on clicking automatically... so basically I can not do anything :(

I am currently able to achieve ~60-70ms reported end to end latency in the server app, which seems about right if I have to judge by the delay between my controllers moving and the video being rendered inside the quest, but the audio itself seems to lag another 50ms behind (at least).
I haven't looked into the audio streaming code as of yet, but given the work currently being done on microphone streaming I'd think you'd already be kinda familiar with it, so here are my question: is this latency something that can be improved upon? I'd expect it to be audio processing pipeline or transfer related rather than a driver limitation... Is audio data transfered as soon as it becomes available or somehow synced or interleaved when outputting video data over UDP?

Wonder if I am the only one, but v3 didn't connect at all. Steam kept saying no headset present. This v4 version does connect but keeps moving the camera non-stop and is unplayable/unusable as it makes me dizzy. v2 still the best so far.

EDIT 2: Updated the title since the issue and solutions in this thread have deviated to the packet loss issue rather than the frame drop issue. (Which are two separate issues)

I've been using the original authors ALVR alpha 5 for the past two days, but because the controller jitter and misalignment was getting too annoying, I searched and found a solution in this fork.

Now that preface is important because original alpha5 worked flawlessly with the exception of the controller issue.
H.265, 50Mbps, 200kb buffer, audio.

With experimental v6 the controller issue is resolved.
Right after the start of the app and server, everything is fine except for maybe a little head-tracking stutter. However, after letting it run for 5~10 minutes, the server is still reporting 72 frames, but the client frames drop down to around 40.
Same settings as above.

Seems like an issue with the mobile app.

Wifi is 1300mbps 5Ghz with 40Hz band. I live at the edge of the forest so there's no congestion. My dedicated 5Ghz access point for the Quest is the only 5Ghz in town.

EDIT: Clarified versions.

Every time I try to launch the app on the headset it gives an "Update in Progress" message and the app does not launch. Other apps including sideloaded ones do work.

The trigger buttons are only registered as simple push buttons, not as analog axes.

The original ALVR did implement it, but it got lost with the cutting of FreePie

Some Oculus exclusive games like Defector or Asgard's Wrath won't play smoothly on ALVR (other examples that run great: Robo Recall, Lone Echo). There is a quite moderate jittering in the headset tracking (not overcome by marking "-1" on the controller jittering section of the PC client).

Please don't hesitate to close this issue if you think it's not relevant.

I'm testing from yesterday. I have to admit it works better than original ALVR. Big congrats.

There is one thing that is missing from original ALVR and that is fix for players height. It was in Other>Apply Offeset in original ALVR. I'm plaing sitting low on couh so Oculus Quest is detecting that I'm just above the ground. In Steam VR home I'm a head laying on the ground or even below it. Can You consider adding this functionality back?

Hey would it be possible to add microphone streaming to this? That would be freaking awesome if possible.

Hey JackD83,

i just want to give you some positive Feedback.
Im playing SteamVR games via ALVR (original) for a couple of weeks now.
Today i tested your ALVR experimental 3 and it works really well!
With the fixed default resoultion and the seperated render resolution from video resolution the output on the quest seems to be much sharper than in the original ALVR and games run very smooth and well for me.
Also the controller fixes are nice too, however there is a bit of a delay in tilting them but i guess it will get better with every new release.. :D

Some short questions:
Would you recommend using "Suppress Frame Drop" and/or "Force 60Hz"?
And what is "Disable send throttling" for?

Thanks for this fork and keep up the great work!
(Win10 Pro 1809 64bit, Nvidia GTX 1070, Driver 430.64)

after 20 mins it seems like it stops streaming all together.
so there is a still image of what i last saw and blackness when i look away from that image. anyone knows why?

After the first launch, the sound settings can not be changed in the UI

Hello All,

since the oculus connect 6 we know that the Quest is capable of 90Hz refresh rate.
You have already built in a switch for 60Hz. I wonder if you could do the same for 90Hz?

ALVR is using a UDP socket to the HMD to stream data and transmit control messages.
This should be split into one TCP connection for signaling and one for streaming

Currently, its possible that the network is loosing packages and the the messages to send I-Frames also gets also lost.

This would be a bigger change

Hi,
Do we need any specific feature when we buy router or any 5ghz router is compatible ?
Thanks :)

Hello,

When running the Experimental V6 release, the client will say "Press Connect Button" to continue. On the server end, it finds the headset and starts the server successfully, but only shows 1 packet going through and all the activity sets to 0.

I reverted to v5 and everything works fine.

Hi,

I'm playing No Man's Sky. You can use the left joystick to move around. When I hold the joystick forward to walk forward, it walks forward as expected. But when you keep holding it forward and you hold the controller upside-down, your character starts walking backwards as if it flips the controls.

I asked my friend who was playing with me and using Virtual Desktop to do the same, but for him those controls don't flip around so I assume this is a wrong implementation in ALVR.

This is really annoying because when you reach for your Analytic Visor, you reach next to your ear and hold the controller upside-down for a second, breaking your forward movement.

ALVR allow us to use the high bandwidtch USB C connection to use the quest as a PC VR headset

can do it?

I had an issue after disconnecting devices from my pc while it was off, replugging everything and starting pc where the virtual cable used for mic streaming didn't receive sound anymore. I dug into the issue by looking into the virtual cable in device manager. I found out that the device instance id was different to the one I found in alvr config file. I then pasted the id from device manager to the config and mic streaming worked again although the id had changed once again after I closed alvr. The mic streaming worked even after it had changed by itself. This could explain why the mic streaming didnt work on the very first time I tried it until I reinstalled a bunch of other virtual adapters I had. There has also always been something about the device and many others I have "not migrated due to partial or ambiguous match." I've searched for solution to that problem before but it can't always be fixed. Cable output has been the default mic in windows settings the whole.

Hello Team,

It is possible that ALVR is not using any NEON video decoding optimization in client side?
I have searched the source code but didn't find anything related to it.
The greatest lag come from the client side decoding, it would be nice to lower this lag through optimization.
This is an av1 example, but it has great potential:
https://medium.com/@ewoutterhoeven/how-arms-neon-enables-efficient-av1-decoding-on-mobile-5fcb3a4f6e7f
Thank you!

because of oculus making the oculus link that comes out in November will be able to take advantage of the oculus link software and improve alvr ? also will you keep working on alvr

Hello, thank you for the great work! I am using latest experimental v5 version. When I enable microphone streaming after a few minutes transport and decode latency increases and it becomes unplayable (see the pic.). My PC is connected with ethernet cable and I share wifi with a 5 Ghz usb dongle. When I disable microphone the problem is gone.

i got alvr v4 but i have a big latency problem when mic steaming is off i get 70-80 ms as normal as soon as i turn on the mic streaming it goes to 80000000 ms is this the software or my hardware i have a belkin ac1200 at 5ghz with a i5-3470 16gb of ddr3 1600 ram with a gtx 1050 ti lp

Hi,

I have installed the latest server and client Experimental v6 and when I open in on the quest it opens the Steamvr world without the menu etc and then closes.

The server says client is connected and it was working fine last night.

Thanks

The MIN_IDR_FRAME_INTERVAL constant in alvr_server/IDRScheduler.h currently is 2 000 000 (2 seconds).
When packetsloss occurs while streaming fast moving picture the picture gets corrupted ("bluring" artifacts appear) for up to 2 seconds (and very often full 2 seconds).

I have done some experiments with decreasing this value to 200 000 (200 ms), 100 000 and 40 000.
I am not sure whether 40 000 was a bit worse than other values, but 100 000 and 200 000 seemed to work very well.
At least in my setup it looked like glitches happen around as often as with 2 000 000, however picture is restored much faster (usually in less than half second).

Maybe this value should be decreased (to 100 000 or 200 000)?

P.S. Looks like upstream have changed the logic in this commit:
polygraphene/ALVR@db6f6bb

When starting DCS World, the game loads up on the desktop fine but SteamVR just hangs on the loading screen in the Quest.

This seems to be an issue with Alpha5, specifically, as apparently it worked fine in A4 (polygraphene/ALVR#395)

I've noticed the haptic feedback of the controllers is really soft: much softer than it would be in the oculus menu or when streaming the same game using SteamVR and another streaming app (for example Virtual Desktop Streamer).

I also noticed that the haptic feedback sometimes starts drifting and the latency of the feedback increases. But when I restart ALVR on the headset or simply press the Oculus button twice to go out and in theALVR streamer app it resets the latency back to no perceived delay.

Whenever I try to start the server I get the following error:

Failed to start audio capture. Thread aborted before starting to loopback capture. message=IAudioClient::Initialize failed: hr = 0x88890008

I have already tried to set the system sound to 16bit,48000Hz(DVD) as suggested in other issues but it didn't help. Unchecking steam sound helps. I'm on experimental v6.

Greetings,
Avencore

Any chance that this fork can be put up as a unique ALVR variant on SideQuest? Especially since the original now seems to be abandonware

I have to redo the room setup for SteamVR every time I launch. Is there some way you can pass the Quest's Guardian data into SteamVR the way that Virtual Desktop does it?

Issue
After Oculus Quest updates to software version 9.0 ALVR might disappear from the Library of installed applications.

Solution
Re-enable Developer Mode in Oculus phone application and reboot Oculus Quest device.

This is not a problem with ALVR, however I assume some people might think so and hopefully they will notice this issue on GitHub.

First of all, I am not programmer so if what I ask or suggest sounds extremely stupid please be nice with me lol.

So as we all know, pretty much every single video it's "filtered". I am not sure which filter it is, bilinear, trilinear, I guess that may depend on each player. My concern it's that I feel vr would benefit from no filtering at all. I am talking about the filter applied to played videos so if you stream at less than 100 % everything it's extremely blurry. This would come handy for FFR too. If you have tried some games on Quest, the FFR makes everything pixelated around edges but I feel that this is much pleasant than a blurred view since that produces eye strain and fatigue.

The ideal case would be having different filtering options, from smoother ones to sharpen ones and even no filter at all. I am not sure whether this would have a negative hit on performance at time of the client decoding. One would guess that not having to filter would alleviate things but, again, I am not programmer. Either way I think that a not filtered video would deliver a better experience. This is specially noticiable when you try to see the steam desktop view from vr, you can't read anything even using 150% of resolution and sending the video at 150MB/s.

I am also aware that no filter could introduce aliasing but I am fine with that, it would be optional after all.

Aditionally, as suggested above, FFR (which I couldn't try yet due buggy release) could have it's own filtering options too on the ALVR side. I have read that there is a couple of blurring filters for later on give a pass with a sharpen one. While it looks good, some of us may prefer to have it unfiltered if we gain some sharpness on the sweet spot.

I guess the main issue here it's that, while it looks good on a flat screen, in vr any bit of blurriness just starts to cause fatigue and eye strain. There is a similar effect on games on CV1 that used an extremely aggressive FXAA and turning it off was more pleasant for several people.

OK now it's when you guys comment and explain why my suggestion it's extremely silly 😋. I am really sorry if my suggestion only worked to clutter the page in such case I could delete it. Thank you for your hard work Btw 😉.

Can't say I fully understand the tech behind FFR but would appreciate any pointers to setting the option values for a reasonably high PC. i7 8700K and gtx1080ti

Hello,
I finally implemented my FFR algorithm on ALVR, mostly. Here's my explanation of the algorithm (with link to an online demo), and here's the code for the server and the client. I'm not opening a pull request because I did not implement the UI and network interop for the ffr variables, instead they are hardcoded here and here.

The two new free variables are foveationStrengthMean and foveationShapeRatio.
foveationStrengthMean is a factor that controls the linear "size" (or square root of the area) of the foveated region. In reality, for how I built the algorithm there is no boundary to the foveation area, the compression is progressively higher the further you look from the center of the screen, so foveationStrengthMean should be interpreted only as a multiplicative term. The higher the value the stronger the foveation effect, the smaller the encode/decode latency. A 5 correspond rougly to Quest's foveation level HIGH.
foveationShapeRatio controls the shape of the foveation (ratio between horizontal and vertical dimensions). It should match the visual acuity graph of the human eye for a for tracked foveated rendering and a bit larger for fixed foveated rendering, because statistically the eye moves more horizontally than vertically. A value between 1.5 and 2 is ok.
The center of the foveation is fixed to the center of the each screen, which is calculated using the field of view reported by oculus vrapi.

A big chunk of the code I wrote (the filtering part) is still not functioning. I tried to fix it but I'm out of time. I thought I could still share what I achieved so far, expecially since the benefit of the filtering would be relatively small (I predict a increase of foveationStrengthMean from 5 to 6 given the same visual quality).

Test:

Oculus Quest, codec h265, bitrate 30Mbps, video resolution 100%(2880x1600), buffer size 200kB.

FFR off (foveationStrengthMean = 0):
encode latency: ~7ms
decode latency: ~11.9ms

FFR on (foveationStrengthMean = 5):
encode latency: ~4ms
decode latency: ~7.2ms

total save: 7.7ms

Hey, I just tried today the brand new experimental v5 and I'm having around 100ms latency just on the controllers. Everything else is OK, and some returning sound from my Quest microphone as well.

how do i stop from hearing my mic in vr???

First of all, thanks to you and to Polygraphene for such awesome work.

I’m playing lone echo. After LOTS of hours in game, in different sessions, I think I isolated a strange behavior of the app. After a random amount of time, It starts losing packets, especially with steam audio when there are loud sounds. This strange “state” in my opinion is completely caused by the app. These are my arguments:

1. If I restart the app, it returns to completely fluid, without any packet drop, for another random amount of time.

2. When the app is in this “packet loss” state, if I press the right oculus menu button, and then click on “resume”, the visor image is black. If I resume the app when it has no packet loss, it resumes completely fine.

3. In this “packet loss” state, even restarting the app is difficult. If I close it and then restart, the visor says that the app was unexpectedly closed, after some flickering.

For completeness, my system is pretty high-end. GeForce RTX2070 heavy oc’d, cpu i5 8-series (six cores), 16 gigs ram, solid disks.

Networking side, I have a high end 5Ghz Linksys router dedicated to ALVR, 2 mts distance, and I’m the only one in the room. On steamvr, I disabled reprojection and I’m at 100% res on SteamVR and on ALVR.

Before entering the strange “packet loss” state, ALVR app and server are OUTSTANDING, steady 72fps, without artifacts of any kind.

Hope this details can help you isolating the problem