Wrong timestamps with some AVI files about ffms2 HOT 8 CLOSED

From fredrik.mellbin on October 04, 2009 13:00:55

Try this compile and report the results. It's another case of a file with NVOPs and
it should now detect those frames properly.

Attachment: ffms2.dll

from ffms2.

From fredrik.mellbin on October 04, 2009 16:30:56

Owner: fredrik.mellbin

from ffms2.

From [email protected] on October 05, 2009 01:31:11

Not really working well. It has two problems:

1. It's still one frame off (one frame too late)
2. It drops frames like crazy. The old version didn't do this.

After taking a closer look, it seems like this version drops a frame every time there are B-frames when
using packed bitstream. So for a IBPBPBBP sequence it drops three frames since there are three "B
groups".

Also, I tested remuxing the file to MKV, using mkvmerge. The resulting file plays back fine and has
correct timestamps, but ffms fails on it in the exact same way.

from ffms2.

From fredrik.mellbin on October 05, 2009 03:01:07

I'm curious, which encoder did you use for that file? I don't know of any that uses
both nvops and b-frames at the same time.

from ffms2.

From [email protected] on October 05, 2009 03:28:55

Just plain old VFW Xvid, and this is something I see Xvid doing very often, whenever
there's a static scene (typically black or non-scrolling credits).

In fact, if you feed that video to Xvid, using its default settings, it'll do the very
same thing again (tested with 1.2.1, but I don't think that has changed in ages).

from ffms2.

From fredrik.mellbin on October 05, 2009 08:28:56

Unpack your bitstream and try again. The nvop detection should work properly then.
What doesn't work is having nvops AND packed b-frames (or that's what I think) and
there doesn't seem to be any way around it except to actually parse the bitstream
thoroughly.

from ffms2.

From [email protected] on October 05, 2009 11:45:27

I figured this out :)

These "strange" frames aren't NVOPs at all. They are just B-frames. It just happens that skipping a block in a P-
frame implicitly skips it in adjacent B-frames. So if a P-frame has all skipped macroblocks, there's nothing to put
into the B-frame, and it ends up very small (8 bytes total in this case).

Let's see an example. Video frames, in display order:
I0 B1 B2 P3 B4 B5 P6

Coded order:
I0 P3 B1 B2 P6 B4 B5

Packed bitstream:
I0 P3B1 B2 N P6B4 B5 N

Nothing strange so far. However, if the frames are unchanging, the B-frames will have every macroblock coded as
"implicitly skipped", which takes 0 bits. This leads to a frame with a size of just 8 bytes.

The good news is that it's very easy to tell B-frames from N-frames apart, since just after the header (000001B6)
comes two bits which determine the frame type. Long story short, I-frames are 0, P-frames and N-frames are 1, B-
frames are 2, and S-frames (basically P-frames with global motion compensation) are 3. N-frames are used to signal a
flush of the last packed P-frame, and are just uncoded P-frames (there's other bit which tells whether the frame is
coded or not, but it's not in a fixed position - still easy to read though if it's really needed).

Examples:

00 00 01 B6 10 60 31 82 3D ... coded I-frame
00 00 01 B6 51 E0 13 FF FF ... coded P-frame
00 00 01 B6 90 E0 64 BF coded B-frame, all blocks skipped
00 00 01 B6 91 60 64 BF coded B-frame, all blocks skipped
00 00 01 B6 92 E0 64 BF coded B-frame, all blocks skipped
00 00 01 B6 51 CF uncoded P-frame (N)
00 00 01 B6 52 CF uncoded P-frame (N)
00 00 01 B6 54 4F uncoded P-frame (N)
00 00 01 B6 57 4F uncoded P-frame (N)

But what does all of this have to do with our problem? It gets better - apparently, ffmpeg's MPEG4ASP decoder has an
special "optimization" - it detects all-skipped B-frames like the ones above, and skips decoding entirely (see
h263.c, line 3196), but that only gets triggered on some B-frames and only when using packed bitstream (haven't
checked it closely, but it seems like it kills the B-frame which is packed together with the P-frame [B1 and B4 in
the example], probably since that's the only place the "resync" code is used). Now I understand why ffdshow-vfw
returns some garbage frames with these files.

Unpacking works with both versions (the newer one is probably killing the B-frames but that's OK since they're
unchanging anyway). What a mess. Can you fix it? :)

It's either detecting a handful of special cases, or unpacking in the fly. Unpacking might be easy, I think the only
thing you have to do is, for P-frames (000001B6 followed by either 01 or 11 binary), search for a B-frame startcode
within the same frame (000001B6 followed by 10 binary), remove that from the frame and hand it in as a new frame.
Then, if you have handed one, just kill the next unencoded P-frame you encounter (000001B6 followed by 01 binary).
Some considerations: there might be other stuff before the 000001B6 header (but that should be only in I-frames, so
it shouldn't matter). The encoders signal the use of packed bitstreams with a special header, but I don't think lavc
is bothered by that (it isn't in my tests). I think the 000001 pattern can't appear inside the bitstream itself,
it'll get escaped, so doing this should be safe. (And yes, I've checked that a unpacked bitstream matches the algorithm I described above)

from ffms2.

From fredrik.mellbin on December 15, 2009 09:25:58

Status: Fixed

from ffms2.