RTMP to WebRTC fps drop by DJI mini 3. about srs HOT 6 CLOSED

I also have the same problem, but I tested with OBS Studio and found the following:

If I stream using the advanced settings and change the Output mode to "Advanced," then:

FFmpeg VAAPI H.264 --> Linux Chrome: OK, Android Chrome: OK
NVIDIA NVENC H.264 --> Linux Chrome: FPS drops (max 1-2 frames per second) - very laggy, Android Chrome: OK

I was able to reproduce the issue with a simple ffmpeg command:
Dropped frame rate: ffmpeg -re -stream_loop -1 -i clip2.mp4 -vcodec h264_nvenc -c:a copy -f flv rtmp://<host>
Correct frame rate: ffmpeg -re -stream_loop -1 -i clip2.mp4 -c copy -f flv rtmp://<host>

It seems that more expensive drones (or controllers) use another algorithm, which is not FFmpeg VAAPI H.264.

Are there any settings I can try to change? Unfortunately, I cannot change anything on the drone's side (no any option in the official DJI app), only on the SRS.

Tested on Ubuntu 22 with the default config, docker

from srs.

Please use h.264 profile baseline if need converting RTMP to WebRTC. Dup to #4052

from srs.

@winlinvip Do you mean in ffmpeg.transcode.conf, because it will be the same as HTTP-FLV, there will be the same delay?

from srs.

Maybe it is the web browser don't support h.264 b-frame, because baseline profile don't support b-frame, that's the reason baseline works well.

@zeldan you can verify by disable the b-frame by -bf 0. I don't have a NVidia gpu, so I can't verify, the h264_nvenc, which is NVidia hardware encoder, must be encode the video frame to h.264 main/high profile with b-frames.

ffmpeg -re -stream_loop -1 -i clip2.mp4 -vcodec h264_nvenc -bf 0 -c:a copy -f flv rtmp://<host>

or just use baseline profile.

ffmpeg -re -stream_loop -1 -i clip2.mp4 -vcodec h264_nvenc -profile:v baseline -c:a copy -f flv rtmp://<host>
the params, -profile:v, is libx264's params, I don't know whether the h264_nvenc support it or not.

from srs.

Sorry, I reproduced the Stuttering or laggy effect, the b-frame is laggy + part rect screen blur effect.
How to calculate or check webrtc fps? did anyone knows?

Here is my way to reproduce the Stuttering effect?

1. ./objs/srs -c conf/rtmp2rtc.conf (don't forget to set CANDIDATE env);
2. publish a video with profile high: ffmpeg -re -stream_loop -1 -i test.mp4 -vcodec h264 -profile:v high -acodec copy -f flv rtmp://localhost/live/livestream
3. use ffmpeg to do transcode, publish the transcoded stream to live/livestream_ff
ffmpeg -i rtmp://localhost/live/livestream -vcodec h264 -profile:v baseline -acodec copy -f flv rtmp://localhost/live/livestream_ff
   (@dancixx don't use ffmpeg.transcode.conf, this cmd will have same effect, the ffmpeg.transcode.conf will result in avalanche effect, the transcoded stream will trigger another transcode process)
4. Now, we have two streams: live/livestream and live/livestream_ff, live/livestream is profile high and have b-frames, while live/livestream_ff is profile baseline without b-frame. You can verify by ffprobe:
   ffprobe rtmp://localhost/live/livestream
ffprobe rtmp://localhost/live/livestream_ff
ffprobe -show_frames rtmp://localhost/live/livestream
ffprobe -show_frames rtmp://localhost/live/livestream_ff (check the pict_type: I, B, P)
5. ffplay live/livestream and live/livestream_ff, the videos play well.
6. check the webrtc http://localhost:1985/rtc/v1/whep/?app=live&stream=livestream_ff, this stream's profile is baseline, but video seems laggy.

If I do transcode in step 2, don't do next steps.
ffmpeg -re -stream_loop -1 -i test.mp4 -vcodec h264 -profile:v baseline -acodec copy -f flv rtmp://localhost/live/livestream
then play webrtc http://localhost:1985/rtc/v1/whep/?app=live&stream=livestream, the video seems fine.

from srs.