Video stream remains with bad quality after a short network degradation about com.unity.webrtc HOT 23 CLOSED

Our team investigated this issue but have not solved it.
Sorry for waiting, but we need more time to fix it. We raised the priority of this matter and working on it.

from com.unity.webrtc.

I appreciate your report in detail.
OK, I am tackling it.

from com.unity.webrtc.

As you say, I found strange results. It might be wrong in the implementation.
I am digging into the native code.

Sofware encoder

Hardware encoder

from com.unity.webrtc.

I fixed the issue and now we are testing this PR.

from com.unity.webrtc.

@WaltzBinaire

However it seems now that in the beginning the bandwidth ramps up unnecessary slow for up to 20 seconds as if it started at zero.

We also couldnt get the bitrate settings to work

I am investigating them.

from com.unity.webrtc.

The originally reported bug is now fixed: I can confirm that the image quality is restored 10-20 seconds after the network quality has gone from bad to good. This is in version 2.1.0-preview.4. Thanks, @karasusan .

from com.unity.webrtc.

Yes, I agree that this issue should be closed, because the originally reported bug has been fixed. 🏅

from com.unity.webrtc.

Thank you very much for looking into this. Let me know if I can do anything to help (for example testing).

from com.unity.webrtc.

Hi. I have some more test results. Without knowing the details of the code, it looks to me as if the HW encoding use much less bandwidth than is available (except for when the bandwidth is very high). The original resolution of the video stream is 1280x720.

Software rendering, good bandwidth (20 Mbps+):

Software rendering, 2Mbps:

Software rendering, 0,78Mbps:

Hardware rendering, good bandwidth (20 Mbps+):

Hardware rendering, 5 Mbps bandwidth. This looks like a bug (the image should not be as blurry as this on 5Mbps ?? ) :

Hardware rendering, 2 Mbps:

0,78 Mbps is not much worse than 2 Mbps, and that is a bit surprising:

from com.unity.webrtc.

@karasusan This bug is still appearing. I have also reproduced it on Linux, using HW encoding of the video. Unity Render Streaming version 2.0.2-preview, and WebRTC version 2.0.2-preview.

Could this bug be indirectly fixed by #118, by setting the quality/bitrate explicitly high on the server at regular intervals? Or are you planning any other solution? I see you are working hard on many other features and bugfixes; however, I really hope you will find the time to look more into this. Intermittent network degradation is normal for many users, so this bug will give very many users a very bad video quality.

Please let me know if I can help, for example by testing new versions or testing specific scenarios.

from com.unity.webrtc.

I am investigating this issue and I noticed the issue might be caused by libwebrtc made with google.
I checked the video quality using two of google chrome via WebRTC.
(VP8 codec is no problem, This issue is only reproduced with H.264 codec)
The issue is reproduced using two of chrome. There is a possibility of a cause is the receiver side.

It is easy to check it if use the sample app "apprtc".

example:
https://appr.tc/?vrc=H264&audio=false&hd=true

Google chrome is using openh264 as h.264 encoder. Our package is using NVIDIA Codec SDK instead.
The algorithm of the bitrate control which used by these implementations should be same.
I will investigate a little more.

from com.unity.webrtc.

Thank you for following up, @karasusan .

I tested the sample app "apprtc" between Chrome on macOS and Chrome on Windows 10, and the video quality is restored normally after about 10 seconds when my network goes from bad to good. So I cannot reproduce this bug with Chrome-only test.

Also, to investigate further on my end, I tested with Safari running on an iPad as the "client" to my Unity "server":

• When running against my Unity server (Unity WebRTC 2.0.5-preview) on Linux using hardware encoding, I never get good video quality when my network goes from good to bad and back to good (the same result I got when testing on Windows server using HW encoding).
• But if I use the same Safari on an iPad to run against my Unity server (Unity WebRTC 2.0.5-preview) on macOS using software encoding, the video quality is gradually improver back to normal quality 10-30 seconds after my network goes from good to bad and back to good (so this works as expected).

from com.unity.webrtc.

@eirikwah
Thanks for your cooperation.

the video quality is restored normally after about 10 seconds when my network goes from bad to good.

Really? Did you set the parameter vrc=H264 to the URL on the chrome?

As you say, this issue is occurred with only Hardware encoder(H.264) not software encoder(VP8, VP9).

from com.unity.webrtc.

Really? Did you set the parameter vrc=H264 to the URL on the chrome?

Yes, I re-checked now and used this URL both on Chome on macOS, and on Windows: https://appr.tc/r/247315345?vrc=H264&audio=false&hd=true. I start with good network, then restrict it to 200 kbps, then wait for 10-15 seconds for the video to switch over to low bandwith (it freezes a short while), then I make the network good again, and after 10-15 seconds the video quality is good again to normal quality. Using Chrome "Version 84.0.4147.125 (Official Build) (64-bit)" on both macOS and Windows.

It would be great if Unity WebRTC would support hardware video encoding with codec VP8 (in addition to H264). We use a "Tesla M60" GPU, which supports HW encoding of VP8. Let me know if there is a way for me to test this (I checked, and it looks like it is unsupported in Unity WebRTC now.)

Another thing I could try is software encoding on Linux, but that is also unsupported in Unity WebRTC now. So currently my backup-plan is to switch from Linux to Windows on our servers, only to be able to do software video encoding. (But I really hope that we don't have to, as this has negative impact on cost and operations for us.)

from com.unity.webrtc.

@karasusan We just tested 2.1 and we couldn't find any improvements with regards to the artefacts in hardware encoding. Also changing the bitrate parameters doesn't change the amount of artefacts. Which NVidia drivers are you using?

from com.unity.webrtc.

@WaltzBinaire
This fix is not included in 2.1.0-preview.1. We will release 2.1.0-preview.3 soon.

About the issue of the bitrate parameters, Could you check it with software encoder?
It might be a bug which occurs with HW encoder.

The package depends on the nvcodec vesion 9.0,
so NVIDIA driver version R418 or later is no problem.

from com.unity.webrtc.

Hey @karasusan - thank you very much for your amazing efforts ! We woudl have two questions.

1. could you describe the error you fixed in 210preview 3 - it is very interesting to us.
2. could you let us know when this fix is out. We believe it is super important as unity Webrtc has the same bug since april - and if those artefacts are gone would be a huge game changer in quality of experience!

again thank you very much! we will test the software encoding!

from com.unity.webrtc.

I published the WebRTC package 2.1.0-preview.4 and uploaded the template project on GitHub Release.
https://github.com/Unity-Technologies/UnityRenderStreaming/releases/tag/2.1.0-preview

could you describe the error you fixed in 210preview 3 - it is very interesting to us.

OK, I will add items about fix to the CHANGELOG

from com.unity.webrtc.

Cross posting, I think this issue is more related:
#164 (comment)

Generally it's way more stable.
The bitrate seems to slowly get up over a long period of time.
Couldn't find in the samples the right place to change the bitrate.

from com.unity.webrtc.

Hey it really seems as if #164 is related - latest preview is a massive jump in quality ! Thank you guys. However it seems now that in the beginning the bandwidth ramps up unnecessary slow for up to 20 seconds as if it started at zero. The frequent bumps however seem to be completly gone. We also couldnt get the bitrate settings to work

from com.unity.webrtc.

Can confirm, quality seems to be bad for about 15-20 seconds and then goes back to normal.

from com.unity.webrtc.

hey @karasusan -where you able to find out why the connection ramps up so slowly? thank you!

from com.unity.webrtc.

#100 (comment)

However it seems now that in the beginning the bandwidth ramps up unnecessary slow for up to 20 seconds as if it started at zero.

@WaltzBinaire created an issue about that.
#182

We also couldnt get the bitrate settings to work

I created an issue.
#183

I am glad if you post information about these issues.
I guess the original issue is already done, may I close this one? > @eirikwah

from com.unity.webrtc.