Add support h265cvbrMaxIprop from IQ profile about majestic HOT 18 OPEN

Just to clarify on the subject.
Constant bitrate is very important for video transmission over the air (for FPV systems for example), where radio channel bandwidth is limited.
Even when the integrated(total) video bitrate per second is within the radio bandwidth, these spikes/"bursts" are over the capacity of the channel, which results in frames arriving delayed and at irregular intervals at the receiver.
This causes video lagging, then quick speeding up, then lagging again...
Here is an example:
https://drive.google.com/file/d/116LI2pddlAL9z5MnkOQQcH1fS0vK13vY/view?usp=sharing

Seems that this value should be set to the HiSillicon encoder sdk structure VENC_PARAM_H265_CVBR_S.u32MaxIprop :
https://github.com/OpenIPC/openhisilicon/blob/0344062f1e255c7e0ee58745868bca7dd4c3cadf/include/comm_rc.h#L336

typedef struct VENC_PARAM_H265_CVBR_S {
	GK_U32 u32MinIprop; /* RW; Range:[1, 100]; the min ratio of i frame and p frame */
	GK_U32 u32MaxIprop; /* RW; Range:[1, 100];the max ratio of i frame and p frame,can not be smaller than u32MinIprop */
	GK_S32 s32MaxReEncodeTimes;
	...
};

In other HiSillicon encoder implementations this is coded like this:
https://github.com/openhisilicon/HIVIEW/blob/720e6f0904e57c698d11e2a5d2aded2e4803e1e2/mod/mpp/3516e/src/scene_auto/src/sample/scene_loadparam.c#LL2514C14-L2514C14

from majestic.

Indeed, it is not implemented yet (and I don't even know when can we have fully implemented HiSilicon scene profiles) but let's test your hypothesis first

I've created a sandbox for simple get/set specific encoder params, set h265cvbrMaxIprop to 100 there and let's check on real device will it help resolve bitrate issues.

In case if you don't have Goke includes I've prepared a binary that should be run after Majestic starts. The same way, you may quickly test other similar options to narrow which ones we should have in backlog
gkrcparams.tar.gz

from majestic.

Unfortunately my gokeV200 cam is waiting for a new flash to resurrect :) so I decided to test on HiSilicon.
Built the OpenIPC image for hi3516ev300, then used its folders like this.

TPATH=$(HOME)/src/openipc-firmware/output/host/bin
MPP=$(HOME)/src/openipc-firmware/output/build/hisilicon-opensdk-c033009fa4551650480604dababcb5a852983593

When running make this gives me cannot find -lgk_api
On the hi3516ev300 camera the precompiled file is not working:

Error loading shared library libgk_api.so: No such file or directory (needed by ./gkrcparams)
Error loading shared library libvoice_engine.so: No such file or directory (needed by ./gkrcparams)

Built the OpenIPC image for goke7205v200, but can't find libgk_api.so there.

from majestic.

Change

-LDLIBS=-lgk_api -lupvqe -ldnvqe -lvoice_engine -lsecurec
+LDLIBS=-lmpi -lupvqe -ldnvqe -lVoiceEngine -lsecurec

I've added a branch with adapted code for HiSilicon

from majestic.

Tried to compile the code, mpi_venc.h and hi_type.h were not found, So I supposed I was missing some HiSilicon SDK.
Found and downloaded this repo
Gave me: no member named ‘stParamH265CVbr’; did you mean ‘stParamH265Cbr’
Changed it and it almost compiled :) , but then I've got this error:

arm-openipc-linux-musleabi/bin/ld: cannot find -lsecurec
collect2: error: ld returned 1 exit status

Removed -lsecurec and it compiled.
But when I run it on the device got this:

If you have time, please attach a prebuild binary to test on hi3516ev300, or I'll have to wait for a week to test on GokeV200

from majestic.

-lsecurec is obligatory, the library can be taken from OpenIPC distro

There is a precompiled binary for ev200:
hircparams.tar.gz

from majestic.

I confirm the precompiled binary starts on ev300, when majestic is running.

I did not notice change in I-frames size on still image in front of the cam, there are still "spikes":

In fact when I start moving the cam so that the image in front of it constantly changing, the bitrate looks much "smoother".

My theory is that since H265 is Variable Bit Rate by nature, when set to CBR, the encoder logic tries to use all the "available" bandwidth by filling it up with the maximum possible size of the I-frame (by adjusting its compression & size)
So on still cam, there is no image changes between I-frames, few P or B-frame are sent, and there is much room left from the total bandwidth for the I-frame.
When the cam is moving, there are much more changes that need to be encoded and they "eat" the bandwidth left for the I-frame.
So I think the value of h265cvbrMaxIprop should be set to lower value to limit the size of I-frame. I'll have to learn to compile your sample to test my idea.
I compile a buildroot image for Hi3516ev300 and get this header files, but no libraries

The first Goke sample compiles but fails with:

Changed LDFLAGS=-L$(MPP)/libraries and LDLIBS=-lgk_api ... still the same

from majestic.

So.
Found the libraries libmpi.so, libupvqe.so in the OpenIPC local build and referenced them:
LDFLAGS=-L$(HOME)/src/openipc-firmware/general/package/hisilicon-osdrv-hi3516ev200/files/lib
Still the compile fails with:

/home/home/src/openipc-firmware/output/host/bin/arm-openipc-linux-musleabi-gcc -I/home/home/src/openipc-firmware/output/build/hisilicon-opensdk-c033009fa4551650480604dababcb5a852983593/include  -L/home/home/src/openipc-firmware/general/package/hisilicon-osdrv-hi3516ev200/files/lib  gkrcparams.c  -lmpi -lupvqe -ldnvqe -lVoiceEngine -lsecurec -o gkrcparams
/home/home/src/openipc-firmware/output/host/bin/../lib/gcc/arm-openipc-linux-musleabi/8.4.0/../../../../arm-openipc-linux-musleabi/bin/ld: warning: libdl.so.0, needed by /home/home/src/openipc-firmware/general/package/hisilicon-osdrv-hi3516ev200/files/lib/libmpi.so, not found (try using -rpath or -rpath-link)

And I can't find libdl.so.0 nowhere in the OpenIPC distro.
Can you give me some clue what I'm doing wrong?

from majestic.

This must be a warning, not a real error (the issue is we're using uclibc-based SDK on top of musl libc in OpenIPC)

from majestic.

I couldn't compile it with the Buildroot image - got another error:

/home/home/src/openipc-firmware/output/host/bin/../lib/gcc/arm-openipc-linux-musleabi/8.4.0/../../../../arm-openipc-linux-musleabi/bin/ld: /tmp/ccyIYgc7.o: in function `main':
gkrcparams.c:(.text+0x1c): undefined reference to `GK_API_VENC_GetRcParam'
collect2: error: ld returned 1 exit status

So I linked to this repo
and compiled. The program now runs on hi3516ev300, only when majestic is also running. I added some more params output.
The settings persist between separate runs, but get reset when the majestic is started again. I'm not sure the changes in the parameters are taken into account by majestic when it is already running...
An interesting feature is that when I try to change a param to an invalid value, I get a notification in majestic's log, like this:

The value of u32MaxIprop seems to be set to 20 by Majestic after each start.
But indeed changing "on the fly" from 1 to 100 and back does seem to affect the video, though not so much as I hoped. But the size of I frame is reduced.

from majestic.

Majestic doesn't use this API calls at all, it may be default params after SDK inits. Anyway test sample with custom encoder params should be started only after majestic loads and inits encoder

from majestic.

I confirm this approach works.
Added extra options to be able to change more parameters of the stParamH265CVbr.
Here are the default values:

stParamH265CVbr.u32MaxIprop = 20 , was 20
stParamH265CVbr.u32MinIprop = 1 , was 1
stParamH265CVbr.u32MaxQp = 10 , was 10
stParamH265CVbr.u32MinQp = 2 , was 2
stParamH265CVbr.bQpMapEn = 10
stParamH265CVbr.s32MaxReEncodeTimes = 51, was 51
stParamH265CVbr.u32MaxQpDelta   = 0, was 0
stParamH265CVbr.enQpMapMode = 51
stParamH265CVbr.u32ExtraBitPercent = 0
stParamH265CVbr.u32LongTermStatTimeUnit = 0

I found out that setting u32MaxQp=30 and u32MaxIprop=2 makes the codec stream much more "smooth" and without spikes and included this change in my FPV setup.

#This will reduce I-Frame size and slutter
/etc/gkrcparams --MaxQp 30 --MaxI 2

Here are my changes. It would be nice to integrate such a tool in OpenIPC build for HiSilicon/Goke cams, to be able to fine tune Majestic stream.
Tested on gk7205v200 and hi3516ev300.

from majestic.

Can you test another SDK provided approach to smooth bitstream as well? It should "distribute" I-frame between different P-frames, but it may add more latency

        VENC_INTRA_REFRESH_S stIntraRefresh = {
            .enIntraRefreshMode = INTRA_REFRESH_ROW,
            .u32RefreshNum = chn->size.height / chn->framesInGop,
            .bRefreshEnable = HI_TRUE,
            .u32ReqIQp = HI_FALSE,
        };
        ret = HI_MPI_VENC_SetIntraRefresh(chn->venc_chn, &stIntraRefresh);

After the refresh function is enabled, the SDK automatically refreshes I-macroblocks by line (from top to bottom) or by column (form left to right) based on u32RefreshNum starting from the first frame of each GOP. The refresh interval is group of pictures (GOP).

When the network bandwidth is limited (such as the wireless network), the intra-P-frame refresh can reduce the heavy network traffic caused by large I-frames, reduce network transmission delay, and decrease the probability of network
transmission errors.

from majestic.

I implemented this call and committed to my fork.
It does affect video. The default values of the params are:

When I set EnableRefresh to true

the video stream "flattens" almost perfectly.

But there are some artefacts over the video at each I-frame, which disappear if I increase the value of u32RefreshNum, but then the video has "spikes" again.
It needs more testing, thanks for the clue.

from majestic.

Do you use H264/H265 slices for each frame? It also can get better quality results

from majestic.

Well, I admit I do not understand your question. The video encoder is set to H265.
I simply call GK_API_VENC_GetIntraRefresh and GK_API_VENC_SetIntraRefresh.
Anyway, the change of h265cvbrMaxIprop to 2 and MaxQP to 30 seems to have positive effect on the video smoothness:
https://www.youtube.com/watch?v=D0_y5k-Vc_s
So the problem seems to be solved, thank you very much!
Still, it would be very nice to support this approach of fine tuning HiSilicon/Goke video encoder via its specific params outside of Majestic with a custom tool like this one.

from majestic.

What is a set of settings we should implement finally?

from majestic.

I changed these to improve the video:

stParamH265CVbr.u32MinQp 
stParamH265CVbr.u32MaxIprop

So if you meant to implement these setting in majestic, these would be enough for my specific case.
Of course having all the properties of stParamH265CVbr configurable will be better.

Since the specific params available for HiSilicon SDK in comm_venc.h are much more, I would suggest that an other approach is to support the repo you provided as an "external template" in openIPC, so that anyone can easily customize it and tweak params he needs, like I did.
This will be "outside" of majestic and will be used only for HiSilicon/Goke cams only when custom param tuning is needed.

from majestic.