ffmpeg with `h264_nvenc` fails to run on gVisor with `-nvproxy` about gvisor HOT 10 OPEN

We don't support graphics/video capabilities yet.

from gvisor.

@ayushr2 we may take on the work to add the video capability to NVProxy. Many of our customers are running into this limitation when seeking to do GPU-accelerated ffmpeg stuff. Do you have any thoughts or objections before we do?

from gvisor.

Please see #10856 which needs to happen before non-CUDA ioctls can be added to nvproxy.

from gvisor.

Thanks for the reply @EtiennePerot. I've made regression testing the first task under our internal project 👍

from gvisor.

We may be able to reuse gVisor's existing ffmpeg image to avoid creating yet another Dockerfile for this. A regression using it can be as simple as this.

from gvisor.

Sounds good. Thank you for letting me know.

from gvisor.

A friendly reminder that this issue had no activity for 120 days.

from gvisor.

@thundergolfer We are aligning internally around how to proceed with adding non-CUDA support. Let me get back to you once we have fleshed out the details.

from gvisor.

how to proceed with adding non-CUDA support

It'd be the NVIDIA Video Codec SDK that we'd need to support, right?

Please do keep us in the loop :) We'd slotted in this work for mid-September but will of course adjust if it doesn't fit with your plans.

from gvisor.

Hi,

As per #10856, nvproxy cannot currently accept patches for nvenc/nvdec commands until it supports NVIDIA capability segmentation. @ayushr2 and others have started to work on this and we expect this to be done (at least structurally done, i.e. the nvproxy ABI definitions will support being tagged by driver capabilities) by early october.

This is a bit later than your planned date for starting this. So in the meantime, as part of this work, it would also be great if you could contribute some NVENC/NVDEC regression tests as well, even if broken in gVisor at PR merge time. This is necessary not just for correctness, but also to ensure long-term maintainability as the NVIDIA driver and userspace libraries change. ffmpeg's h264_nvenc can take care of exercising nvenc, so that should definitely be one such test. Is there something similarly simple we can use for nvdec?

from gvisor.