Uniform time representation about pyav HOT 10 CLOSED

It gets even more complicated, since there is a global time_base, as well as one per Stream, codec context, and Packet.

from pyav.

Since we are creating something for the vast majority of use cases, but not all, I think we can drop the ultimate precision that rationals give us, and go with something reasonably precise, like seconds stored as doubles.

There must still be a time_base set on the stream/codec/global/wherever, and you must pick a reasonable value for it to not lose precision, but that is true of rationals as well.

We still get 53 bits of integer precision, which I think is close enough to what the AVFraction gives you for practical purposes.

So... doubles for the user. Make it so?

from pyav.

Since a timestamp without a timebase is useless, I propose we create a timestamp class that combines them and expose all pts, dts, durations and ts related variables to python through it. I was thinking of something like this.

cdef class TimeStamp(object):
    cdef int64_t _ts
    cdef lib.AVRational _time_base

    def __init__(self, ts not None, time_base not None):
        to_avrational(time_base, &self._time_base)            
        self._ts = ts

    @staticmethod   
    def from_seconds(seconds, time_base=None):
        cdef TimeStamp ts = TimeStamp.__new__(TimeStamp)

        if time_base is None:
            ts._time_base.num = 1
            ts._time_base.den = lib.AV_TIME_BASE
        else:
            to_avrational(time_base, &ts._time_base)
        ts.seconds = seconds
        return ts

    def __float__(self):
        return self._ts * float(self._time_base.num) / float(self._time_base.den)

    def __str__(self):
        return str(float(self))

    property seconds:
        def __get__(self):
            return float(self)

        def __set__(self, value):
            self._ts = value * float(self._time_base.den) / float(self._time_base.num)

    property ts:
        def __get__(self):
            return self._ts
        def __set__(self, value):
            self._ts = value

    property time_base:
        def __get__(self):
            return avrational_to_faction(&self._time_base)

        def __set__(self, value):
            cdef lib.AVRational rat
            to_avrational(value, &rat)
            # Scale timestamp to new timebase                
            self._ts = lib.av_rescale_q(self._ts, self._time_base, rat)
            self._time_base = rat

from pyav.

I disagree.

As soon as you start doing operations in different timebases without care, you can easily overflow even the huge precision that rationals give you. So a naive rational time will very quickly fail. (Imagine combining badly conceived (as they often are) timebases for standard and NTSC variants of 30 and 24fps; you immediately hit a denominator past 64 bits).

I also think that this will generally end up with less precision than floating time as ration time with a timebase that is relatively prime to a stream's timebase will have larger rounding errors than a double will.

For all practical purposes, doubles have way more than enough precision, and for nearly all purposes I bet the encoded time values will come out exactly the same.

Ergo, I'm still in the camp of floating time in seconds (therefore without a base).

Mike

On Jul 13, 2014, at 12:43 PM, Mark Reid [email protected] wrote:

Since a timestamp without a timebase is useless, I propose we create a timestamp class that combines them and expose all pts, dts, durations and ts related variables to python through it. I was thinking of something like this.

cdef class TimeStamp(object):
cdef int64_t _ts
cdef lib.AVRational _time_base

def __init__(self, ts not None, time_base not None):
    to_avrational(time_base, &self._time_base)            
    self._ts = ts

@staticmethod   
def from_seconds(seconds, time_base=None):
    cdef TimeStamp ts = TimeStamp.__new__(TimeStamp)

    if time_base is None:
        ts._time_base.num = 1
        ts._time_base.den = lib.AV_TIME_BASE
    else:
        to_avrational(time_base, &ts._time_base)
    ts.seconds = seconds
    return ts

def __float__(self):
    return self._ts * float(self._time_base.num) / float(self._time_base.den)

def __str__(self):
    return str(float(self))

property seconds:
    def __get__(self):
        return float(self)

    def __set__(self, value):
        self._ts = value * float(self._time_base.den) / float(self._time_base.num)

property ts:
    def __get__(self):
        return self._ts
    def __set__(self, value):
        self._ts = value

property time_base:
    def __get__(self):
        return avrational_to_faction(&self._time_base)

    def __set__(self, value):
        cdef lib.AVRational rat
        to_avrational(value, &rat)
        # Scale timestamp to new timebase                
        self._ts = lib.av_rescale_q(self._ts, self._time_base, rat)
        self._time_base = rat

—
Reply to this email directly or view it on GitHub.

from pyav.

Responding to @markreidvfx's comment in #32:

There is so much precision in a double, that unless you are at the absolute extremes of possibility (way out of reason) you will identify your time with more precision than the time_base.

I've been doing a bunch of modelling of time for a project at @FluentImage where we have gone through this exact problem, and it has also been demonstrated to me that there are NLE's that use floating time.

At the far end of practical:

• For a fixed 1 million fps you need only 32 bits of the 53 bits of integer precision a double gives for a year long clip.
• For nanosecond precision over a year you need 38 bits.
• Hitting the Nyquist frequency of visible light requires 50 bits.

Sure, you can't do everything that is possible with it, but for all real purposes it will likely result in less errors than rounding with av_rescale_q.

A time_base must still be sensibly chosen, but it doesn't have to beat you over the head all of the time. And we can always retain (or re-expose) the raw_pts properties if you do want full control (we just keep them to the minimal locations required instead of exposing copies of time_base on every object).

from pyav.

Works for me! I vote for doubles too then. Seconds is easier to understand anyway.

from pyav.

Hooray!

Okay, so:

1. all of the timing research from #32 should be compiled into the dev docs;
2. we should establish what is the minimum number of time_base(s) (I think just the stream, and maybe for the codec descriptor (that I don't think currently exists));
3. a number of precision preserving functions for converting to/fro;
4. make it go!

from pyav.

I'm inclined to leave the pts/dts/time_base the way they are, but add the floating representation in time. Then we have all the complexity, or all the simplicity, depending on what you feel like/need at the time.

from pyav.

I think the way things are handled at the moment are fine : we reflect pts / dts / time_base exactly, and provide the convenience time for lazy users :)

from pyav.

Since this was written, I feel the time handling has gotten quite a bit nicer.

I'd still be keen to see time (as a float) be settable. But I don't ever do it because I don't want to decide how to warn the user that it could be a bad idea due to loss of precision due to a bad time_base (not that floats are bad on their own... there is an overkill amount of precision there for realistic cases).

So... if you're reading, you have floating time. If you're writing, you're understanding what is going on. I'm good with that. Lets drop the issue count by one more.

from pyav.