libav does not decode correct sample block size about faudio HOT 25 CLOSED

If it’s older than 4.0 it’s too old.

from faudio.

At the moment I can only test patches to FAudio directly; because of the licensing issues blocking us this is pretty low on the TODO list since there is other stuff I need to get working that doesn't have a patent blocking it.

For reference, here's our xwma test program: https://github.com/FNA-XNA/FAudio/tree/master/utils/testxwma

from faudio.

Couple quick changes to zero out the uninitialized memory (for sanity) and clean up some of the sample-vs-frame math that was going on:

1afcf01...35c55e4

This somewhat unintentionally fixes one case i found where the offsets were only off by 1... for a stereo stream. Not sure how that happened, but hopefully it won't happen again now that this is in.

from faudio.

From Johan in the Discord...

I did some more debugging and I've found that the FFmpeg WMA decoder does not seem to be sample accurate. Sometimes it produces more and sometimes less samples per packet than specified in the seek table. And because we're using the seek table to calculate the expected number of samples in the buffer, this causes us to think we're missing samples at the end (or to not play the entire buffer). Looping/seeking is also going to be inaccurate but that might not be as noticable.

So that... kind of explains why it's not 100% consistent each time? In any case, this is a pretty big problem; we've fixed our end of it but if we want this to work properly it'll have to be fixed in FFmpeg.

from faudio.

Possible idea to explore: Take our existing WMA test program and add a DumpXAPO that's attached to the source; the idea is what we can run that XAPO against both XAudio2 and FAudio and compare the samples decoded (ideally not resampled) by both libraries. From there we can use XAudio2's output as a control and compare various attempts to fix this in FAudio and FFmpeg.

from faudio.

I believe this is the same issue...? https://trac.ffmpeg.org/ticket/7473

from faudio.

@flibitijibibo
I built some of the samples from the Microsoft SDK here: https://github.com/walbourn/directx-sdk-samples/tree/master/XAudio2 and after some tweaks i got them to work with wine.

I also built libfaudio from latest git, and tried compiling with ffmpeg aswell (Ubuntu 18.04 libavcodec-dev 7:3.4.4-0ubuntu0.18.04.1), but are not able to get any sounds during the WMA/XWMA part of the XAudio2BasicSound.exe demo.

0009:trace:xaudio2:IXAudio2Impl_Release (0x114848)->(): Refcount now 1
Playing mono WAV PCM file...
Playing mono WAV ADPCM file (loops twice)...
Playing 5.1 WAV PCM file...
Playing mono xWMA file...
Playing 5.1 xWMA file...
Finished playing
0009:trace:xaudio2:DllMain (0xf7ad0000, 0, 0x1)

Is it my ffmpeg "too old", or is there a issue?

You can download the binaries i tested with here: https://github.com/SveSop/wine_patches/blob/master/xaudio.tar.xz

Build log of libfaudio w/ffmpeg: https://launchpad.net/cybermax-dexter/+archive/ubuntu/sdl2-backport/+build/16519929/+files/buildlog_ubuntu-bionic-amd64.faudio_19.03-bionic5ppa2_BUILDING.txt.gz

from faudio.

@flibitijibibo
Updated my PPA to use ffmpeg 4.1.1, and xWMA mono sample for XAudio2BasicSound.exe demo plays fine!
However, xWMA 5.1 bugs out with:
[wmapro @ 0x7dcb6600] Reserved bit is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.

[wmapro @ 0x7dcb6600] Reserved bit is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[wmapro @ 0x7dcb6600] If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/incoming/ and contact the ffmpeg-devel mailing list. ([email protected])
[wmapro @ 0x7dcb6600] num_vec_coeffs 3800 is too large
[wmapro @ 0x7dcb6600] num_vec_coeffs 2792 is too large
[wmapro @ 0x7dcb6600] Channel transform bit is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[wmapro @ 0x7dcb6600] If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/incoming/ and contact the ffmpeg-devel mailing list. ([email protected])

If you want whole log, let me know.

from faudio.

I think the error message speaks for itself...

from faudio.

@flibitijibibo
Not sure what you mean by "speaks for itself" other than "Doh.. just upgrade ffmpeg you tool".
This is the actual wav file the xaudio2 sample plays: https://github.com/walbourn/directx-sdk-samples/blob/master/Media/Wavs/musicsurround_xwma.wav

Things i have tested:
Ubuntu 18.04 w/ffmpeg-4.1.2 play the file fine with ffplay musicsurround_xwma.wav
Windows 10 w/ffmpeg-4.1.1 play the file fine with ffplay musicsurround_xwma.wav
wine-devel(GIT c7f3231) w/ffmpeg-4.1.1 play the file fine with ffplay musicsurround_xwma.wav

Compiled latest faudio w/ -DFFMPEG=ON and ffmpeg-4.1.2 (ppa here: https://launchpad.net/~cybermax-dexter/+archive/ubuntu/sdl2-backport/+packages )
XAudio2 demo mentioned earlier fails with the above error.

Ofc i could have made the packages totally wrong, but since ffmpeg (ffplay) plays the file just fine when used for playing, i would kind of believe ffmpeg library is able to handle this? Or am i totally mistaken?

I admit i am a total newb, and might just aswell have made (several) errors on my package setup. Tips are welcome :)

PS. Or should i post this as a separate issue if you deem it warrants a "issue" to work on?

from faudio.

I'll translate it:

Reserved bit is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented. If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/incoming/ and contact the ffmpeg-devel mailing list. ([email protected])

Channel transform bit is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented. If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/incoming/ and contact the ffmpeg-devel mailing list. ([email protected])

TL;DR: This is FFmpeg's issue, not ours.

from faudio.

ffplay -loglevel 48 musicsurround_xwma.wav does not show any error with reserved bit, nor channel transform, so it must be some weirdness that XAudio2 does i guess.

Could it be a wine bug?

from faudio.

No. It's either an FFmpeg issue (which I just said...) or the WAVEFORMATEXTENSIBLE data needs to be looked at. You would need to look at what the samples generate for those files.

from faudio.

[NULL @ 0x7f3f78000b80] Opening 'musicsurround_xwma.wav' for reading
[file @ 0x7f3f78001740] Setting default whitelist 'file,crypto'
Probing mp3 score:1 size:2048
Probing xwma score:100 size:2048
[xwma @ 0x7f3f78000b80] Format xwma probed with size=2048 and score=100
[xwma @ 0x7f3f78000b80] Before avformat_find_stream_info() pos: 184 bytes read:32768 seeks:0 nb_streams:1
[wmapro @ 0x7f3f78002640] extradata:
[wmapro @ 0x7f3f78002640] [10] [0] [0] [0] [0] [0] [0] [0] [0] [0] [0] [0] [0] [0] [e0] [0] [0] [0] 
[xwma @ 0x7f3f78000b80] stream 0: start_time: -192153584101141.156 duration: 8.759
[xwma @ 0x7f3f78000b80] format: start_time: -9223372036854.775 duration: 8.759 bitrate=134 kb/s
[xwma @ 0x7f3f78000b80] After avformat_find_stream_info() pos: 147658 bytes read:147658 seeks:0 frames:27
Input #0, xwma, from 'musicsurround_xwma.wav':
  Duration: 00:00:08.76, bitrate: 134 kb/s
    Stream #0:0, 27, 1/48000: Audio: wmapro (b[1][0][0] / 0x0162), 48000 Hz, 6 channels, fltp, 48 kb/s
[wmapro @ 0x7f3f78003840] extradata:
[wmapro @ 0x7f3f78003840] [10] [0] [0] [0] [0] [0] [0] [0] [0] [0] [0] [0] [0] [0] [e0] [0] [0] [0]

From the xwma.c file:

 /* xWMA shouldn't have extradata. But the WMA codecs require it,
         * so we provide our own fake extradata.
         *
         * First, check that there really was no extradata in the header. If
         * there was, then try to use it, after asking the user to provide a
         * sample of this unusual file.
         */

FFmpeg decodes and plays it, but something else might happen when "its coming from xaudio2".

from faudio.

@flibitijibibo
The error is caused by 5.1 surround sound XWM(a) files. I have converted a couple of different WAV files to XWMa format with Microsoft xWMAEncode.exe, and it seems as when there is a 5.1 surround sound XWMa file, this bug happens.

The XWMa file plays just fine with ffplay.

2 channel or mono XWMa files also play just fine, so something strange must happen when Xaudio2 sends this to libfaudio i think.

I am not really sure how i should make a bugreport of this to FFmpeg team, since it is working fine tho. Do you want to experiment with the files? Attaching a .gz of the original .wav and the converted _xwma.wav.
Demo_wavs.zip

from faudio.

This is the spot you care about:

https://github.com/FNA-XNA/FAudio/blob/master/src/FAudio_ffmpeg.c#L116

from faudio.

I have been toying around with this, and all i can really come up with is that when playing with ffplay, "some" of this "waveformatextensible" data is just ignored/cleared, but when FAudio copies this over for ffmpeg to play, there is some data that is "not supported" (ref. error msg).
I gather these lines copies the waveformatextensible data blob over to ffmpeg?
https://github.com/FNA-XNA/FAudio/blob/master/src/FAudio_ffmpeg.c#L138-L142

I have no clue what data this actually contains, or what bits i could/should clear out for ffmpeg to not throw this error.

Commenting out the FAudio_memcpy lines, and just copy some "fake" data with av_ctx->extradata throw a different error and do not crash.
[wmapro @ 0x7dcaed80] bits per sample is 0 is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.

I dont know what av_ctx->extradata[?] actually would contain the settings for "bits per sample" tho.

from faudio.

Yeah, that memcpy should cover everything past the base WAVEFORMATEX structure. But that 'bits per sample' message is really strange, it implies the format doesn't provide a bit depth...? I would dump out the entire WAVEFORMATEXTENSIBLE struct and see what we're being sent in XAudio2 vs what the real header of the file says. Maybe we do have to deliberately throw out some stuff, not sure.

from faudio.

I dont have any great ideas, and i have tried to fiddle a bit with it, but i come up short :)

I wonder tho, is this some sort of conversion? Or does it not come into action when FFmpeg is used?
https://github.com/FNA-XNA/FAudio/blob/master/src/FAudio.c#L419-L422

I only ask, cos it seems as the only usable xWMA files i am able to produce using "wmapro" is multichannel audio, and if some sort of conversion would happen, perhaps there is some added bits from that causing this? (Regular pcm_s16le encoded wav's in 5.1 seems to work fine tho).

from faudio.

I simplified the XAudio2BasicSound.exe file, so that it only loads the wmapro 5.1 wav "musicsurround_xwma.wav", so you can test it if you like?
XAudio2BasicSound.zip

If you run it in Win10 it requires DirectXRedistJune2010 package (For XAudio2), and msvc140 (c++ redist).

from faudio.

@flibitijibibo Oh, i was not aware of the testxwma "player" :) Nice!
Compiled and used ./testxwma musicsurround_xwma.wav and that caused the same error as XAudio2BasicSound.exe does with "reserved bit" and "channel transform bit".

Atleast another testproggy to fiddle with in my spare time. Thx :)

from faudio.

With the latest revision the decoding responsibility has been moved to GStreamer. We still have an issue when moving from one WMA buffer to the next, but this was present in the previous decoder as well.

from faudio.

At the moment I can only test patches to FAudio directly; because of the licensing issues blocking us this is pretty low on the TODO list since there is other stuff I need to get working that doesn't have a patent blocking it.

I don't understand, shouldn't WMA1/WMA2 US patents be expired by now if the formats were released in 1999?

from faudio.

The patent portfolio that covers the various formats and their revisions over the years is several dozen pages long with countless companies with various expiration dates. If there was a straightforward date, it probably would have gotten its own website like S3TC did.

from faudio.

Doing a bit of spring cleaning...

Closing as NOTOURBUG - WMA issues with libav should be filed with that project; at this point my only real data is going to come from protonmediaconverter so this is effectively abandoned.

from faudio.