Audio and video codecs about wuffs HOT 6 OPEN

In my research on video codex's I have found mention of this NUT mux format from ffmpeg. Maybe it is a small enough problem to be useful.

Other possible candidates for video is https://www.jonolick.com/home/mpeg-video-writer (256 lines of code!)

References

godotengine/godot-proposals#4710 (has some nut details at the end)

from wuffs.

Audio would be nice. FLAC and MP3 is already listed on https://github.com/google/wuffs/blob/main/doc/roadmap.md

Video is a lot more work, but, yeah, it would be nice to have something like ffmpeg with much more memory safety guarantees.

As for hand-written assembler code, Wuffs' PNG decoder already uses CPU-arch specific intrinsics, which is roughly equivalent.

from wuffs.

As for hand-written assembler code, Wuffs' PNG decoder already uses CPU-arch specific intrinsics, which is roughly equivalent.

Depends on the situation; there are cases where the compiler will do a bad job at stuff like instruction scheduling or register allocation, but I agree that asm should be a last resort. That said, making assembler code safe would be amazing, if perhaps out of scope.

from wuffs.

Has there been any consideration for the API for these decoders.

Audio should be pretty simple, a config for communicating the sample rate, amount of channels, and bit depth. And perhaps an extension on the IO buffers to let them be used as ring buffers.

For video I don't think it's as simple as the image API augmented with audio. But I have no real experience with movie clip APIs to know what one should expect from them. Especially when you want to be able to use GPU acceleration for playback.

from wuffs.

For video I don't think it's as simple as the image API augmented with audio. But I have no real experience with movie clip APIs to know what one should expect from them. Especially when you want to be able to use GPU acceleration for playback.

I consider GPU acceleration to be out of scope for this.

from wuffs.

Has there been any consideration for the API for these decoders.

I haven't done any API design work yet. I was going to do it at the same time I started on an actual FLAC or MP3 decoder, but there's other work to do first.

from wuffs.