Comments (7)
Here's a simpler test case, H.264 this time. There are five frames, IBIBI in presentation order. If you request frame 4 (0-based) and then frame 2, you get frame 4 twice.
works.mp4
in the same archive doesn't have the problem, even though it contains exactly the same video. broken.h264
and works.mp4
were created by x264-r3095-baee400.exe --open-gop
, and broken.mp4
was created by ffmpeg -i works.mp4 -c copy broken.mp4
(ffmpeg 5.0.1).
from ffms2.
Reproduced. It is however caused by FFmpeg marking all I-frames as IDR-frames on stream copy. Most files in the world could be secretly broken. Have a nice evening.
from ffms2.
Given that:
- The number of affected files is probably very large as you said
- Even if it's FFmpeg's fault, the whole point of ffms2 is to work around FFmpeg's problems and get accurate frame seeking by hook or crook
- The problem still exists in the current head (3f69ac5) unless I'm missing something
I think this issue should be left open. That way it's easier to find if someone else runs into the problem, and maybe someone will even submit a patch for it, even if you never work on it. If it must be closed, at least close it as wontfix instead of completed.
from ffms2.
AFAIK, there is no reasonable way to support / work around this, since, for example, the stss
box in those MP4s mark non-IDR I-frames as IRAP.
Or well, there is no way to work around it unless FFmpeg decides to expose more than I, P, and B as frame types in its API, since this is the root cause of both the remuxing issue, and an inability to work around the bad remuxes. The reason broken.h264 and the mkv is busted is the same (they rely on the parser).
There might be an open bug on FFmpeg Trac (as this is an FFmpeg iissue), but I didn't find one.
from ffms2.
I did encounter this bug when googling yesterday: https://trac.ffmpeg.org/ticket/8820
Basically it implies that at some point FFmpeg sometimes implicitly leaked which frames are truly seekable to when skipping. I failed to reproduce it even with older builds and then proceeded to inelegantly poke FFmpeg's internals and the parsed list of stss keyframes. Which is, as we've stated, is indeed different between the clips and FFmpeg does use
from ffms2.
I looked at this a bit and I'm not sure that marking I-frames as key frames is the problem.
According to this answer, FFmpeg only marks IDR-frames and recovery-point I-frames as key frames. The I-frames in broken.*
are marked as recovery points with recovery_frame_cnt = 0
, which means, I think, that you can seek there and correctly decode every frame that's later in presentation order. So having a key frame there seems like a good thing.
With Elephant's Dream, the problem is that FFmpeg returns the packets in decoding order with no information about the correct presentation order. So FilePos
, OriginalPos
, FrameType
, and KeyFrame
in FFMS2's index are all wrong, and I'm impressed that it still manages to return the right frame some of the time.
Muxing to mp4 or mkv using FFmpeg doesn't solve the problem because FFmpeg writes the same bogus PTSs to the muxed file, and then trusts them when reading it later, so you end up with the same index. This is definitely a bug in FFmpeg unless I'm missing something.
Muxing to mkv using MKVToolNix does solve the problem. It generates its own seemingly correct timestamps, the FFMS2 index is correct, and I wasn't able to reproduce this issue. Remuxing it with FFmpeg produces another working file, so the problem is just the initial generation of the timestamps. The file generated by MKVToolNix has cue points for all three I-frames (even though only the first is IDR), and all three I frames are marked as key frames in FFMS2's index, but it works anyway.
ffprobe -show_frames
lists the frames in presentation order, so it is possible to get that information from the ffmpeg libraries. But it's very slow. It seems to be completely decoding the video even though it only displays a few basic properties of it. I don't know whether it's possible to get the information from FFmpeg without the speed hit.
With broken.*
, the index is correct (ignoring the key-frame issue), but for some reason, when attempting to seek to frame 2 and decode it, although the seek succeeds and the appropriate data seems to be going to avcodec_send_packet
, avcodec_receive_frame
ends up failing with AVERROR_EOF
. Frame 2 is an I-frame and a recovery point, and we're only trying to decode that frame, so I can't see any reason why it shouldn't work. With works.mp4
, because frame 2 isn't marked as a key frame, decoding starts from frame 0 instead, and that ends up working. So I'm not convinced that marking frame 2 as a key frame is the problem in this case either. It's somehow triggering another problem.
from ffms2.
So I'm not convinced that marking frame 2 as a key frame is the problem in this case either. It's somehow triggering another problem.
Internally seeking is (nowadays) simply done with av_seek_frame(FormatContext, VideoTrack, Frames[n].PTS, AVSEEK_FLAG_BACKWARD);
meaning that whatever frame at or before the timestamp FFmpeg thinks is usable will be the target. So if FFmpeg internally has a somewhat correct idea (which it does, hidden deeply inside the mov/mp4 parser) it will still pick an earlier frame and work.
There's a huge number of bugs related to seeking and timestamps in FFmpeg, that's why I kinda gave up and wrote https://github.com/vapoursynth/bestsource instead which just linearly decodes everything. Snappy results on most things up to 30 min long on modern hardware.
from ffms2.
Related Issues (20)
- Framerate guessing code may be obsolete
- VC-1 decoding not frame-accurate unless decoding from beginning. HOT 1
- video is unseekable HOT 1
- FFMS2 can't be build because of relocation error HOT 3
- Set up permissions for Github Workflows
- Problem if the file path has some special character HOT 2
- cross-compile ffms2 not quite working, need advice HOT 10
- Create a Security Policy HOT 1
- ffmpeg support > 4.x HOT 1
- Infinite Loop in GetAudio for certain files HOT 4
- Hash pin GitHub actions HOT 1
- FFVideoSource: No video track found HOT 4
- Bad field order when using rffmode=1 HOT 6
- Wrong function definition of DestroyIndex in ffms2-api.md
- Fails to index FLAC audio file HOT 7
- Expose `log2_chroma_h` and `log2_chroma_w` in `FFMS_Frame` HOT 4
- why you require such the most latest version? HOT 1
- Desynchronized sound in the FB clip
- `Colorspace` field in `FFMS_Frame` HOT 1
- Outdated version on crates.io HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ffms2.