Migrate RSTP streaming example to Gen2 about depthai-experiments HOT 3 CLOSED

Question about how depthmaps fits into this pattern. Is it considered a problem/challenge format wise to flow depthmaps through the device side encoding pipeline? This lies right at the edge of my understanding and I'm very interested in being able to stream depth over rtsp and to better understand the nuance of the formats as they pertain to HW encoding & streaming and potential tradeoffs that are either manageable or perhaps unavoidable.

Can you comment on these links in terms of describing the ability/quality to stream depthmaps using depthai HW encoding?

https://www.itu.int/wftp3/av-arch/video-site/0510_Nic/VCEG-AA12.doc
https://developers.google.com/depthmap-metadata/encoding
https://mpeg.chiariglione.org/standards/exploration/immersive-video/depth-map-formats-used-within-mpeg-3d-technologies

hoping 1) to learn and 2) that the question even makes sense.
thanks.

from depthai-experiments.

Hi @eric-schleicher,
AFAIK, the videoencoder node supports only a limited number of frame formats and we cannot encode depth on device yet.
Regarding depthmaps support - I think @szabi-luxonis or @themarpe can elaborate but I don't think we'll be able to support custom formats in this node

from depthai-experiments.

The video encoder accelerator usually supports 8-bit NV12 color and 8-bit monochrome input, for all modes: H.264/H.265/JPEG.
So disparity can be encoded (but not in subpixel mode, where it goes up to 12-bit)

(The video encoder also should supports 10-bit input for H.26x, not JPEG, same formats as above, but this may not be configurable yet.)

from depthai-experiments.