qntm / t-a-i Goto Github PK

> tai.leapSeconds.map(l => tai.atomicToUnix(l.atomic));
[ -283996800000,
  -265679999999.99997,
  -252460800000,
  -194659199999.99997,
  -189388800000,
  -181526400000,
  -168307200000,
  -157766400000,
  -152668800000.00003,
  -142128000000.00003,
  -136771200000,
  -126230400000.00002,
  -60480000000.00001,
  63072000000,
  78796800000,
  94694400000,
  126230400000,
  157766400000,
  189302400000,
  220924800000,
  252460800000,
  283996800000,
  315532800000,
  362793600000,
  394329600000,
  425865600000,
  489024000000,
  567993600000,
  631152000000,
  662688000000,
  709948800000,
  741484800000,
  773020800000,
  820454400000,
  867715200000,
  915148800000,
  1136073600000,
  1230768000000,
  1341100800000,
  1435708800000,
  1483228800000 ]

Those values should all be integers. Some kind of improper computation is going on here.

These discrepancies are significant, because if I pass in a Unix time of e.g. -265680000000 for conversion, it ends up falling on the wrong side of a boundary where the TAI/Unix offset changed, and returning an incorrect result.

Worst-case scenario, it may be necessary to pre-compute these values. Hmm.

In the next major version of t-a-i my intention is to stop shipping CommonJS modules and start shipping ES modules instead. As t-a-i actually does seem to be in use, I'm not intending to make a breaking change like this arbitrarily, as I have with all my other, largely unused npm packages. Currently my plan is to wait for Node.js 14 to end Long-Term Support, at the end of April 2023. In May 2023 t-a-i will drop support for Node.js 14, which of course is a breaking change and will require a major version bump from t-a-i@3 to t-a-i@4, so I will take this opportunity to carry out the conversion to ES modules at the same time.

As with t-a-i@1 and t-a-i@2, support for t-a-i@3 will be dropped once t-a-i@4 is released. New leap seconds, if and when they are announced, will be incorporated into t-a-i@4 only. In particular,

• In January 2023, a leap second may be announced for the end of June 2023. If so, this leap second will be added to t-a-i@3.
• In May 2023, t-a-i@4 will appear, and support for t-a-i@3 will be dropped.
• In June 2023, the leap second announced in January 2023 may occur. If so, both t-a-i@3 and t-a-i@4 will recognise it.
• In July 2023, a leap second may be announced for the end of December 2023. If so, this leap second will not be added to t-a-i@3, only t-a-i@4.
• In December 2023, the leap second announced in July 2023 may occur. If so, only t-a-i@4 will recognise it, not t-a-i@3.

As per several sources, such as this IERS listing of historic offsets between TAI and UTC and this table provided by the US Naval Observatory (ftp://maia.usno.navy.mil/ser7/tai-utc.dat), the relationship between UTC and TAI is in fact well-defined as far back as 1 January 1961. As we can (sort of) see from the table, during this earlier period from 1961 to the end of 1971, UTC seconds were very slightly shorter than TAI seconds, so the offset between the two "slid" continuously at varying rates. Both the ratio between TAI and UTC seconds and the absolute offset between TAI and UTC were adjusted manually on several discrete occasions, up until 1 January 1972 when a "final irregular jump of exactly 0.107758 TAI seconds" brought UTC to 1972-01-01 00:00:00 at the exact instant that TAI reached 1972-01-01 00:00:10, and the rest is history.

Unix time can probably be extended backwards to 1 January 1961 too, which means that in theory it should be possible to provide Unix<->TAI conversions during this earlier period.

The spectre of floating point raises its ugly head at this point...

There is a forthcoming proposal for JavaScript date/time handling, Temporal. This proposal is currently at Stage 3. When it reaches Stage 4, I would kind of like it if t-a-i provided some kind of useful integration with Temporal. Here is the documentation for this new API.

This may be an interesting venture because of some possible points of complexity.

• Temporal makes the same choice (mistake?) which many date/time handling APIs do, which is to treat Unix time as the fundamental underlying bedrock of time, from which all time zones are derived. In reality, International Atomic Time (TAI) is a deeper bedrock from which Unix time is derived. This is a basic design decision in Temporal and not something which is ever going to change. So, any integration with Temporal will require t-a-i to misrepresent TAI as a "time zone" derived from Unix time, not the other way around.
• Temporal.Instant represents a fixed point in Unix time, which, under some models of the relationship between Unix time and TAI, is in fact ambiguous. Temporal does offer some mechanisms for resolving ambiguity but I think they don't work in this case because they assume the reverse relationship where Unix time is unambiguous and the time zone is ambiguous.
• Furthermore, Temporal does not acknowledge that certain Unix times never happened. Unlike most "time zones", converting these Unix times to TAI must fail somehow.
• Temporal.TimeZone does not appear to account for the possibility of time zones where the rate at which time passes differs from that under Unix time. Under the "smear" model for leap seconds, Unix time and TAI pass at different rates during inserted or removed time. It's entirely possible to carry out conversions, but the offset between one and the other will be continually changing. What problems does this cause?
  • For timeZone.getNextTransition, for example, there is an implicit assumption that the offset between zoned time and Unix time only changes on certain discrete occasions. During a smear, do we simply return the current instant, because the offset is continually changing? Does this break anything?
  • One workaround for this problem is to simply abandon support for the smear model and use the "overrun", "break" and "stall" models. But this doesn't help us for pre-1972 history. From 1961 to 1971 inclusive, TAI and Unix seconds were simply different lengths, no matter what model one considers.
• Temporal.Instant seems to use nanoseconds, whereas JavaScript Date APIs have used milliseconds up until now. I have some secret scaffolding which allows absolute precision by using arbitrary precision ratios of BigInts internally, so this should be doable, although I am still curious as to what the shape of this "exact" API should be.

Same basic reasoning as for #2 and #3. For Unix block end times, we cannot just use the start of the next Unix block, as there is almost always an offset between the two. Irritatingly, the offset is not a neat number of Unix milliseconds but a neat number of often-slightly-shorter TAI milliseconds, which I suspect eventually yields the same 64-bit float but I won't know for sure until I've crunched the numbers...

The beginning of TAI for our purposes was 1961-01-01 00:00:01.422818000 TAI, which is -283996798577.182 TAI milliseconds. t-a-i calculates this figure internally as Date.UTC(1961, JANUARY, 1, 0, 0, 1, 422) + 0.818000.

However, strictly speaking, this figure cannot be represented exactly as a 64-bit IEEE 754 float. The number actually stored is the closest float to this value, which is -283996798577.1820068359375. This figure comes some 0.0000068359375 seconds before TAI began. This doesn't seem like a huge problem in the scheme of things, but if an instant in time is considered to be on the wrong side of one of these boundaries, then the wrong computation can be applied to it, and a visibly wrong result can come back. In this specific case, what should happen if we attempt to convert this value to Unix time is that an exception should be thrown, because it is too early. What actually happens is that the conversion is carried out with no problems!

Around half of the figures used for TAI block start points prior to 1972 are faulty in this way.

Instead of storing the closest 64-bit float to the beginning of a TAI block, we should store the next lowest available 64-bit float. In this case, the next float is -283996798577.18194580078125, which comes 0.00005419921875 seconds after TAI began. Not all of those decimal places are necessary, the smallest unambiguous representation for JavaScript's purposes is -283996798577.18195.

It is probably also a good idea to use these values explicitly rather than computing them using Date.UTC() and adding a fraction on.

There are at least four models that I am aware of for modelling the relationship between Unix milliseconds and TAI milliseconds during an inserted leap second:

1. The Unix millisecond count overruns during the inserted time, instantaneously backtracks at the end of the inserted time, and then repeats itself.
2. The Unix millisecond count stalls during the inserted time.
3. Unix time is undefined during the inserted time; a TAI->Unix conversion on this interval throws an exception.
4. From midday to midday surrounding the inserted time, Unix time runs fractionally slower than normal. After 86,400,000 slightly-longer-than-normal Unix milliseconds have elapsed, 86,401,000 TAI milliseconds (or so) have elapsed, including the inserted time.

All of these relationships have different advantages and disadvantages. In models (1), (2) and (3), the relationship is not strictly monotonic. In model (1), Unix and TAI are in a one-to-many relationship, which is reflected in t-a-i's tai.oneToMany APIs, which return arrays. In model (2), Unix time truly "ignores" leap seconds but there is still technically a one-to-many relationship here - one Unix time maps to a whole range of TAI times. tai.oneToOne provides access to only the last Unix time in the range. In model (3) basic time conversions during inserted time cause exceptions to be thrown. tai.oneToOne used to behave like this. And in model (4), the relationship is strictly monotonic but Unix seconds vary in length, as they used to do in the bad old days pre-1972 - worse than that, the variation in length isn't even a nice round number. Pre-1972, a Unix millisecond was precisely 13 or 15 TAI picoseconds longer than a TAI millisecond, and t-a-i is designed to take advantage of this precision when performing its internal calculations. With smearing, that's more like 11,574.074074... TAI picoseconds longer. This fraction can't be modelled exactly and will lead to imprecise results without an overhaul of t-a-i's internals and the sacrifice of some guarantees in its output.

None of these models are authoritative and there appears to be no agreement on which is correct, even in POSIX itself, which as far as I can tell ties itself in knots and directly contradicts itself.

So, what I want to do is this:

• t-a-i should provide access to all four models according to user preference, not just (1) and (2) as it does currently.
• t-a-i should name the models something sensible instead of the rather unclear oneToMany and oneToOne models.
• t-a-i's API may need to change somewhat to reflect the fact that model (4) does not allow for precise results. The precise unixToAtomicPicos methods, for example, may just have to be removed entirely, or replaced with something which returns a BigInt numerator and denominator.

There are various ways of modelling the relationship between TAI and UTC over the course of a leap second, and t-a-i offers four of these. For real-world purposes, a smear seems to be the most popular of these options, and t-a-i offers one of these, a strictly linear 24-Unix-hour smear from 12:00 to 12:00 across the discontinuity, favoured by Google. However, there are other possible smears. Meta's 25 July 2022 essay advocating for the abolition of leap seconds - whose topic I'm not addressing here - mentions that they use a 17-hour sinusoidal smear beginning at the moment of discontinuity. Whether or not Meta follows through on their ambitions, this is the model they use internally at the time of writing, so it would be nice to add support for it to t-a-i.

Right now, implementing this would be annoyingly difficult. t-a-i internally uses precise ratios of BigInts in order to ensure precise output - in fact, currently it has "secret" internal APIs which return those exact ratios, with no truncation, for perfect accuracy. The sin function isn't amenable to this approach - in most cases, the result can't be precise. So, what to do? How to modify the API? What level of precision is good enough? Bit of a conundrum. (As an interesting side note, Google rejected sinusoidal smears in part because a linear smear was "simpler [and] easier to calculate".)

The linked article also mentions some other smearing models:

There are also different algorithms on the smearing itself. There is a kernel leap second correction, linear smearing (when equal steps are applied), cosine, and quadratic (which Meta uses). The algorithms are based on different mathematical models and produce different offset graphs:

• We have linear smearing in the bag.
• I have no clue what they mean by "cosine smearing" - that is, I have no clue how they think this differs from sinusoidal smearing.
• "Quadratic smearing" has a few possible interpretations, all of which would be somewhat annoying to implement but not actually impossible with the current implementation, I think, but I'll hold off because this is so vague.
• I also have no clue what they mean by "quadratic (which Meta uses)" when they clearly stated that they use a sinusoidal smear and even provided graphs to demonstrate.
• Finally, I have absolutely no clue which smearing model the chart here is intended to represent. It looks like a real-world capture from a system which wasn't behaving particularly well.

Google's article also mentions several other smears which I might implement someday, but they don't seem to be in active use, so I consider them a lower priority.