Giter Club home page Giter Club logo

boring-expansion-spec's Introduction

Linux CI Status CircleCI Build Status OSS-Fuzz Status Coverity Scan Build Status Codacy Badge Codecov Code Coverage Packaging status OpenSSF Scorecard

HarfBuzz

HarfBuzz is a text shaping engine. It primarily supports OpenType, but also Apple Advanced Typography. HarfBuzz is used in Android, Chrome, ChromeOS, Firefox, GNOME, GTK+, KDE, Qt, LibreOffice, OpenJDK, XeTeX, PlayStation, Microsoft Edge, Adobe Photoshop, Illustrator, InDesign, Godot Engine, Unreal Engine, and other places.

xkcd-derived image

For bug reports, mailing list, and other information please visit:

http://harfbuzz.org/

For license information, see COPYING.

Documentation

For user manual as well as API documentation, check: https://harfbuzz.github.io

Download

For tarball releases of HarfBuzz, look here. At the same place you will also find Win32/Win64 binary bundles that include libharfbuzz DLL, hb-view.exe, hb-shape.exe, and all dependencies.

The canonical source tree is available on github.

The API that comes with hb.h will not change incompatibly. Other, peripheral, headers are more likely to go through minor modifications, but again, we do our best to never change API in an incompatible way. We will never break the ABI.

If you are not sure whether Pango or HarfBuzz is right for you, read Pango vs HarfBuzz.

Development

For build information, see BUILD.md.

For custom configurations, see CONFIG.md.

For testing and profiling, see TESTING.md.

To get a better idea of where HarfBuzz stands in the text rendering stack you may want to read State of Text Rendering 2024. Here are a few presentation slides about HarfBuzz at the Internationalization and Unicode Conference over the years:

Both development and user support discussion around HarfBuzz happens on the github.

To report bugs or submit patches please use github issues and pull-requests.

For a comparison of old vs new HarfBuzz memory consumption see this.

Name

HarfBuzz (حرف‌باز) is the literal Persian translation of “OpenType”, transliterated using the Latin script. It also means "talkative" or "glib" (also a nod to the GNOME project where HarfBuzz originates from).

Background: Originally there was this font format called TrueType. People and companies started calling their type engines all things ending in Type: FreeType, CoolType, ClearType, etc. And then came OpenType, which is the successor of TrueType. So, for my OpenType implementation, I decided to stick with the concept but use the Persian translation. Which is fitting given that Persian is written in the Arabic script, and OpenType is an extension of TrueType that adds support for complex script rendering, and HarfBuzz is an implementation of OpenType complex text shaping.

Packaging status of HarfBuzz

Packaging status

boring-expansion-spec's People

Contributors

anthrotype avatar behdad avatar davelab6 avatar justvanrossum avatar liamquin avatar lorp avatar rsheeter avatar skef avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

boring-expansion-spec's Issues

[`VORG`] >64k expansion

The length of the VORG table is determined from its major/minor version and the subsequent content. The only currently defined version 1.0 can only encode 16bit glyph indices. Here is how we expand the table's definition to encode glyph vertical origin for glyph indices beyond maxp.numGlyphs:

Basically, derive the length of the table from the version number and the table struct, then derive the excess bytes, then follow the same algorithm for deriving advance-width in the hmtx/vmtx expansion: #7 [We should move that to a named data-structure. Trimmed array?]

[`hmtx`/`vmtx`] >64k expansion

I talk about hmtx only. vmtx case is similar.

Currently hmtx table length is determined by hhea/maxp in the following way:

  • hhea.metricDataFormat is required to be 0 for current format;
  • hmtx is required to be 2 * hhea.numberOfHMetrics + 2 * maxp.numGlyphs bytes long.

The rest of this issue describes how we relax the second requirement, to allow encoding advance width of arbitrary number of glyphs in hmtx table, regardless of the value of maxp.numGlyphs.

This is how an advance width is assigned to every glyph index beyond maxp.numGlyphs:

  • Let B be the excess bytes at the end of the hmtx table beyond 2 * hhea.numberOfHMetrics + 2 * maxp.numGlyphs bytes.
  • If length of B is odd, ignore the last byte of B.
  • Treat B as an array of big-endian uint16 advance-width numbers of glyph indices starting at maxp.numGlyphs. For any glyph index that is in range in the font but out of range in this array, use the last item of the array.

[`COLR`] >64k expansion

COLR table, no version bump needed; just new types:

// Paint a non-COLR glyph, filled as indicated by paint.
struct PaintGlyph24
{
  uint8                  format; // = ??
  Offset24<Paint>        paint;
  uint24                 glyphID;    // not a COLR-only gid
                                 // shall be less than maxp.numGlyphs
}

struct PaintColrGlyph24
{
  uint8                  format; // = ??
  uint24                 glyphID;    // shall be a COLR gid
}

struct Clip24 {
  uint24                startGlyphID;  // first gid clip applies to
  uint24                endGlyphID;    // last gid clip applies to, inclusive
  Offset24<ClipBox>     clipBox;   // Box or VarBox
}

struct ClipList24
{
  uint8                 format;  // Set to 2.
  ArrayOf<Clip24, uint32> clips;  // Clip records, sorted by startGlyphID
}

umm... the following requires a COLRv2 though:

struct BaseGlyphPaintRecord
{
  uint16                glyphID;
  Offset32<Paint>       paint;  // Typically PaintColrLayers
};

Maybe postponing this table to next version is fine...

Reference:
https://github.com/googlefonts/colr-gradients-spec

[`maxp`] Relax

Version 1.0 can be used with TrueType fonts when backward-compatibility with legacy systems is required when such systems rejects TrueType fonts with a version 0.5 maxp table.

Otherwise, a version 0.5 maxp table is encouraged.

If maxp table is not present, then maxp.numGlyphs is presumed to be 0.

Note that API functions that currently return maxp.numGlyphs can choose to return other values, like number of glyphs with shapes (derived from loca table) instead, or if loca is missing, number of glyphs derived from hmtx table.

Combined 32bit FormatOffset with more offset bits

This is one of the tricks I thought of as ugly and hence wasn't planning of pursuing. But seeing #1 and other suggestions re varint encoding, I'm going to put it here for consideration.

Currently, all OpenType Lookup subtables (except for (Chain)?Context(Subst|Pos)Format3) start with:

  uint16 Format;
  Offset16 Coverage;

The Format is currently only used to encode 1, 2, or 3. We can reuse the top byte of format for additional offset bits. That is:

  uint8 OffsetTopByte
  uint8 Format;
  uint16 OffsetLow2Bytes

This has the nice property that if the offset fits in 16bits, then Format will be parsable by old implementations. If offset is large, then Format will look wrong by old implementations and ignored.

Data-Types: Base-Encompassing Offsets

Similar to offsets currently used in the font format (big-endian), with varying widths (16, 24, or 32bits). The base is always the start address of the offset field itself. In other words, the offsets are relative.

Will write them as eg. BEOffset16To<Type>.

[`loca`/`glyf`] >64k expansion

Currently head table must have:

  • int16 | indexToLocFormat | 0 for short offsets (Offset16), 1 for long (Offset32).
  • int16 | glyphDataFormat | 0 for current format.

and loca table must be of length (head.indexToLocFormat ? 4 : 2) * maxp.numGlyphs.

By relaxing the latter requirement, we allow encoding glyph shapes for glyph indices beyond maxp.numGlyphs.

loca table is simply treated as an array of Offset16 or Offset32 values as is, just that the number of entries is derived from the number of tables, not the other way around.

glyf table composite glyphs need an extension to allow referring to higher-up glyph indices. That will be done in another issue.

[`glyf`] Cubic Bezier curves

Use the last remaining bit of the simple-glyph flags bits:

  • 0x80 CUBIC Bit 7 Curve is cubic Bezier

The bit is only relevant for off-curve points. For on-curve points the value is reserved.

The number of consecutive off-curve points that have cubic flag on MUST be even. Every two off-curve points define one cubic Bezier.

The implicit-on-curve algorithm will extend to cubics. I'll expand on this in a followup comment or edit.

[`GSUB`] `SingleSubst` >64k expansion

The Coverage table part is covered in #30.

Format 1 delta addition math only affects lower 16bits of the gid. Format 3 delta addition math is module 2^24.

Add two new formats:

struct SingleSubstFormat3 {
  uint16 format; // == 3
  Offset24To<Coverage> coverage;
  int24 deltaGlyphID;
};

struct SingleSubstFormat4 {
  uint16 format; // == 4
  Offset24To<Coverage> coverage;
  Array16Of<GlyphID24> substitutes;
};

[`fvar`] Legalizing HOI

If multiple axes have the same axisTag, at most one must be non-hidden, and such non-hidden axis must come first. There's no further requirements on the ranges and default values of such axes.

[`GPOS`] Spacing Marks

Original proposal Martin Hosken and I wrote in 2016 is here:
https://github.com/OpenType/opentype-layout/blob/63d169c92160832cf5b2333f28b559c80c9a4389/proposals/20151104-spacemark.md

I could not fully implement it in HarfBuzz though. The problem is that it makes a circular dependency between calculating glyph offsets and glyph advances. I need to think it through more to see if those can be solved as a system together. As such, not slating this for the first release milestone yet.

[`post`] Relax

When backward-compatibility with existing implementations is not a concern, either a version 2.0 (with glyph-names) or 3.0 (without glyph-names) can be used, to encode italic-angle and underline-position/thickness.

If none of those are needed, the post table can be omitted. For example, the italic-angle can be encoded in the font's fvar table in a variable font, or the font's STAT table in a font that has one.

[`GDEF`/`GSUB`/`GPOS`] `Coverage` / `ClassDef` tables >64k expansion

Two new formats each:

struct CoverageFormat3 {
  uint16 format; // == 3
  Array16Of<GlyphID24> glyphs;
};

struct CoverageFormat4 {
  uint16 format; // == 4
  Array16Of<Range24Record> ranges;
};

struct ClassDefFormat3 {
  uint16 format; // == 3
  GlyphID24 startGlyphID;
  Array24Of<uint16> classes;
};

struct ClassDefFormat4 {
  uint16 format; // == 4
  Array24Of<Range24Record> ranges;
};

struct Range24Record {
  GlyphID24 startGlyphID;
  GlyphID24 endGlyphID;
  uint16 value;
};

[`GSUB`/`GPOS`] (`Chain`)`Context` >64k expansion

We add one new Context and one new ChainContext format that expand Format1 of each to beyond 64k.

struct ContextFormat4 {
  uint16 format; == 4
  Offset24To<Coverage> coverage;
  ArrayOf<Offset24To<GlyphRuleSet24>> ruleSets;
};

struct GlyphRuleSet24 {
  ArrayOf<OffsetTo<GlyphRule24>> rules;
};

struct GlyphRule24 {
  uint16 glyphCount;
  GlyphID24 glyphs[inputGlyphCount - 1];
  uint16 seqLookupCount;
  SequenceLookupRecord seqLookupRecords[seqLookupCount];
};
struct ChainContextFormat4 {
  uint16 format; == 4
  Offset24To<Coverage> coverage;
  ArrayOf<Offset24To<ChainGlyphRuleSet24>> ruleSets;
};

struct ChainGlyphRuleSet24 {
  ArrayOf<OffsetTo<ChainGlyphRule24>> rules;
};

struct ChainGlyphRule24 {
  uint16 backtrackGlyphCount; 
  GlyphID24 backtrackGlyphs[backtrackGlyphCount];
  uint16 inputGlyphCount;
  GlyphID24 inputGlyphs[inputGlyphCount - 1];
  uint16 lookaheadGlyphCount;
  GlyphID24 lookaheadGlyphs[lookaheadGlyphCount];
  uint16 seqLookupCount;
  SequenceLookupRecord seqLookupRecords[seqLookupCount];
};

Format 2 (Class-based format) is extended to Format 5 just to address offset overflows. Format 3 (Coverage-based format) is NOT extended like that, because it only encodes one rule, so overflows are unlikely. Format 2 and 3 are otherwise addressed in #30.

struct ContextFormat5 {
  uint16 format; == 5
  Offset24To<Coverage> coverage;
  Offset24To<ClassDef> classDef;
  ArrayOf<Offset24To<ClassRuleSet>> ruleSets;
};
struct ChainContextFormat5 {
  uint16 format; == 5
  Offset24To<Coverage> coverage;
  Offset24To<ClassDef> backtrackClassDef;
  Offset24To<ClassDef> inputClassDef;
  Offset24To<ClassDef> lookaheadClassDef;
  ArrayOf<Offset24To<ClassRuleSet>> ruleSets;
};

The RuleSet and ChainRuleSet are NOT extended, because they are class-based, not glyph-based, so no extension is necessary.

[`TupleVariationStore`] Sparse axes

This is the counterpart to ItemVariationStore issue #17, to address similar problems, by using same structures designed in that issue. To integrate, we need to bore through extension mechanisms into existing TupleVariationStore.

[`GSUB`] `LigatureSubst` >64k expansion

The Coverage table part is covered in #30.

Add one new format:

struct LigatureSubstFormat2 {
  uint16 format; == 2
  Offset24To<Coverage> coverage;
  Array16Of<Offset24To<LigatureSet24>> ligatureSets;
};

struct LigatureSet24 {
  ArrayOf<OffsetTo<Ligature24>> ligatures;
};

struct Ligature24 {
  GlyphID24 ligatureGlyph;
  uint16 componentCount;
  GlyphID24 componentGlyphIDs[componentCount - 1];
};

[`GSUB`] `MultipleSubst` / `AlternateSubst` >64k expansion

The Coverage table part is covered in #30.

Add one new format, which are structurally the same across the two (as are the existing format1's):

struct MultipleSubstFormat2 {
  uint16 format; // == 2
  Offset24To<Coverage> coverage;
  Array16Of<Offset24To<Sequence24>> sequences;
};

typedef ArrayOf<GlyphID24> Sequence24;

struct AlternateSubstFormat2 {
  uint16 format; // == 2
  Offset24To<Coverage> coverage;
  Array16Of<Offset24To<AlternateSet24>> alternateSets;
};

typedef ArrayOf<GlyphID24> AlternateSet24;

[`HVAR/VVAR`] >64k expansion; discussion

That was already done, thanks to the work in googlefonts/colr-gradients-spec#319

The way we did it there, there will be a chance of misinterpretting the data by an old interpretter, of a new font if it's variable, has >64k glyphs, uses a VarIdxMap (aka DeltaSetIndexMap).

At the time I was convinced this was not a big deal. I still think it's okay. After all, we're used to newer fonts not working on older implementations. That they are not misinterpretted was always a nice touch, but ultimately a useless features.

[`GDEF`] >64k expansion

Of the GDEF fields:

  • GlyphClassDef is addressed by #30.
  • AttachmentList should be marked deprecated.
  • LigatureCaretList allows specifying ligature carets for up to 64k glyphs, which are specified using a coverage table, which by way of #30 allows glyph indices >64k; so this might be okay as long as fonts are limited to 64k ligature glyphs needing ligature carets.
  • MarkAttachmentClass is also addressed by #30.
  • MarkGlyphSets allows specifying up to 64k different sets, specified as coverage objects, which by way of #30 allow glyph indices >64k, and these are referenced via 32bit offsets so that's great. So as long as the entire font does not need more than 64k distinct sets we are good.
  • ItemVariationStore is extended separately as needed; it does not store glyph indices anyway; and the offset to it is 32bit (good).

That is to say, there is no pressing work to be done here. Definitely nothing for first milestone.

The biggest pressure here is the 16bit offsets; the combined size of the various structures make it hard to fit them all within those offsets.

Fortunately, MarkAttachmentClass's use can be reduced by the compiler in favor of MarkGlyphSets. And MarkGlyphSets and ItemVariationStore already use 32bit offsets, so they can escape far away enough. LigatureCaretList uses one layer of offset16's. So as long as GlyphClassDef for the whole font can be built in about just under 64k, the table is buildable. That probably will become the most pressing point on how to order the glyph-order. But, again, workable for even a whole-unicode font.

As such, no work to be done here for now. Yay!

[`GPOS`] `MarkBasePos` / `MarkLigPos` / `MarkMarkPos` >64k expansion

The Coverage table parts are covered in #30.

Add one new format of each, just to upgrade offsets of the top-level subtable to 24bit. All downstream structs are reused and not expanded.

struct MarkBasePosFormat2 {
  uint16 format; // == 2
  Offset24To<Coverage> markCoverage;
  Offset24To<Coverage> baseCoverage;
  uint16 markClassCount;
  Offset24To<MarkArray> markArray;
  Offset24To<BaseArray> baseArray;
};

struct MarkLigaturePosFormat2 {
  uint16 format; // == 2
  Offset24To<Coverage> markCoverage;
  Offset24To<Coverage> ligatureCoverage;
  uint16 markClassCount;
  Offset24To<MarkArray> markArray;
  Offset24To<LigatureArray> ligatureArray;
};

struct MarkMarkPosFormat2 {
  uint16 format; // == 2
  Offset24To<Coverage> mark1Coverage;
  Offset24To<Coverage> mark2Coverage;
  uint16 markClassCount;
  Offset24To<MarkArray> mark1Array;
  Offset24To<Mark2Array> mark2Array;
};

[`cmap`] >64k expansion

Formats 12/13 already support 32bit glyph indices.

Add format16 subtable, which is exactly like format14, except that whereas format14 uses:

struct UVSMapping {
  uint24 unicodeValue;
  uint16 glyphID;
};

the new format16 will use:

struct UVSMapping {
  uint24 unicodeValue;
  uint24 glyphID;
};

(or uint32, if we choose to go that way instead; but I think 24 is more appropriate).

[`ItemVariationStore`] Sparse axes

Currently, ItemVariationStore follows the TupleVariationStore in listing min,peak,max for every axis defined in fvar table to specify each region, aka tent. This has multiple problems. By switching to a sparse representation, ie. just listing active axes, we save bytes, make axes "free", remove offset-overflow on high number of axes, as well as enable higher-order interpolation by way of repeating same axis index on a as-needed basis.

Surprisingly, ItemVariationStore has very low expansion capacity embedded in it. No version, no reserved bits. There's only one bit I see, and that's exactly in the right place: in the VariationRegionList struct: "The high-order bit of the regionCount field is reserved for future use, and must be cleared."

Currently VariationRegionList looks like this:

struct VariationRegionList {
  uint16 axisCount;
  uint16 regionCount;
  RegionAxisCoordinates variationRegions[regionCount * axisCount];
};

with

struct RegionAxisCoordinates {
  F2DOT14 startCoord;
  F2DOT14 peakCoord;
  F2DOT14 endCoord;
};

So this is how to bore through this:

If the high bit of regionCount is set, the current VariationRegionList is followed by:

  uint8 major; // set to 1
  uint8 minor; // set to 0
  Array16Of<Offset24To<GeneralizedRegion>> moreRegions;

The indices looked up in the moreRegions array are regionIndex - regionCount.

A GeneralizedRegion addresses multiple VarFonts1.0 design problems, and is designed to be reusable from TupleVariationStore as well. Major design elements:

  1. Lists axes sparsely, and allows repeated axis indices, allowing on-demand HOI,
  2. Allows specifying only region peak, or start/peak/end, or start/peak/end as needed,
  3. Allow specifying region shape; whereas VarFonts1.0 only allowed for simple tent shape, allow other shapes,
  4. Allow multiple "regions" to add up. Eg. multiple tents combined into one generalized region, such that the same delta-set need not be repeated. Possibly with different coefficients. #16

GeneralizedRegion is specified in #43.

Unlike VarFonts1.0 regions, when evaluating GeneralizedRegion, there's NO requirement that peak cannot be 0, or that startCoord and endCoord should have the same sign, etc.

Higher Order Curves

Expansion thoughts designers perspective

I love the idea to ditch the CFF and CFF2 tables and put all outlines in glyf.
It means having lineto, quadraticsto and cubiccurveto available, thus allowing me to combine 2nd and 3rd degree segments in one outline. I expressed this wish with explanations in a PS<>TT battle some years ago, that lecture is still online

However I would also really really would like to be able to expand that with fourthdegreecurveto and fitfthdegreecurveto.
Why?
I want to be compatible with product designers and be able to achieve the same curve continuity as products have.
I make a lot of logo-fonts in which outlines need to have a much higher quality than what is possible now, because such logos will be cut in gigantic sizes and live long and prosperous in front of buildings.
I want to be able to draw for instance the outline of an iPhone in a variable font. It is impossible. The connections from curves to straight sides are smooth with a nice increasing curve speed, and it was designed with a 5th degree curve. With 2nd and 3rd degree curves, I can only make rough approximations of such curves, and I need to insert extra points near the connection, that will not interpolate well, it is a total design disaster.

Yes, 5th degree curves need more time to calculate, but I would only use them where really necessary, and at the same time I would replace cubics with quadratics wherever possible and save calculation time.
Adobe has dictated the 3rd degree Bezier for the graphic world, but for drawing beautiful shapes, they really suck.
If a carpenter uses a machine to cut wood from 3rd degree curves, she/he will feel with the fingers that the connections aren’t right, and use sanding paper until it feels good.

5th degree Bezier curves allow feel-good curves in fonts and machines.

[`GPOS`] `PairPos` >64k expansion

The Coverage table part is covered in #30.

Two new formats are added. Format 2 is replicated just to address offset-overflow issues.

struct PosFormat3 {
  uint16 format; == 3
  Offset24To<Coverage> coverage;
  uint16 valueFormat1;
  uint16 valueFormat2;
  Array16Of<Offset24To<PairSet24>> pairSets;
};

struct PairSet24 {
  uint24 pairValueCount;
  PairValueRecord24 pairValueRecords[pairValueCount];
};

struct PairValueRecord24 {
  GlyphID24 secondGlyph;
  ValueRecord valueRecord1;
  ValueRecord valueRecord2;
};
struct PosFormat4 {
  uint16 format; == 4
  Offset24To<Coverage> coverage;
  uint16 valueFormat1;
  uint16 valueFormat2;
  Offset24To<ClassDef> classDef1;  
  Offset24To<ClassDef> classDef2;
  uint16 class1Count;
  uint16 class2Count;
  Class1Record class1Records[class1Count];
};

[`GSUB`] Move Lookup

There was a Move Lookup proposal from Martin Hosken in 2016:
https://github.com/OpenType/opentype-layout/blob/63d169c92160832cf5b2333f28b559c80c9a4389/proposals/20151104-movelookup.md

There are several aspects of that proposal that I didn't fully support back then. Biggest one being that I thought the glyph-skipping part should be left to the Glyph Filtering #25 proposal.

I now have a proposal in mind that I'm satisfied with and I think satisfies Martin's requirements as well, and I try to write that down in a subsequent comment.

What mine does not encompass however, and I like to see if we can make it do, is to be equivalent-in-power to the AAT mort/morx RearrangementSubtable. That subtable can reorder up to two glyphs at each side. Quoting https://developer.apple.com/fonts/TrueType-Reference-Manual/RM06/Chap6morx.html:

Verb Results
0 no change
1 Ax => xA
2 xD => Dx
3 AxD => DxA
4 ABx => xAB
5 ABx => xBA
6 xCD => CDx
7 xCD => DCx
8 AxCD => CDxA
9 AxCD => DCxA
10 ABxD => DxAB
11 ABxD => DxBA
12 ABxCD => CDxAB
13 ABxCD => CDxBA
14 ABxCD => DCxAB
15 ABxCD => DCxBA

Not slating this lookup for first milestone.

varint encoding thoughts

I’m watching Behdad’s presentation on “better engineered font formats”, and finding that a lot of the discussion (in the early parts anyway) are focused on representation of integers in the file format.

File formats of TrueType’s vintage are made largely of integers with hand-chosen fixed size representations. A fixed size representation facilitates random access. But the size chosen induces a Goldilocks problem: too small imposes limits, while too large is wasteful of file size. Simply changing the 16 bit glyph id’s, array lengths, and offsets of OpenType to 32 bits would double the size of those parts of the file.

Varint encodings address this tradeoff, and are widely used. For example, LEB128, aka VByte, is used in protobuf. They are usually thought to be slower than fixed-sized encodings, and are also not thought to be compatible with random access. Both preconceptions indicate a failure of imagination.

On CPU, recent work by Daniel Lemire’s group shows that decompression of sequences of varint encoded integers can happen at very high speeds (~4 billion integers per second). LEB128 decoding is a reasonably simple and straightforward monoid, which means that it can be decoded very efficiently on GPU as well.

What about random access? That’s possible too. Acceleration structures can be built, either computed quickly in a pass over the file, or, optionally, packaged with the file.

If I were building a font file format from scratch, I would seriously consider a layered encoding strategy. The top layer would be defined in terms of a sequence of 32 bit integers. Offsets would count 32 bit values, which I expect will economize entropy. The bottom layer would just be the LEB128 encoding of that sequence, or, perhaps, with packaged acceleration structures as well.

I believe this would be efficient in space and time (and likely compress well with Brotli), but perhaps more importantly would be a conceptual simplification as it removes the need to hand-allocate bit sizes.

I will very likely use this technique in piet-gpu (right now it uses fixed-size encoding, which is wasteful of space).

[`GPOS`] SinglePos >64k expansion

The Coverage table part is covered in #30.

Format 1 needs no change.

Add one new format to address offset overflow issues and value count limitation:

struct SinglePosFormat3 {
  uint16 format; // == 3
  Offset32To<Coverage> coverage;
  uint16 valueFormat;
  uint24 valueCount;
  ValueRecord valueRecords[valueCount];
};

Higher Font Resolution

Expansion thoughts designers perspective 2

The limit of point coordinates in TT-space is currently 2^14. (16384)
I regularly run into this limit. I have made quite a few fonts on 4096 UPM already, when extra detail is important. But then a glyph can not be wider than 4 times the type size, and in fonts with logos or icons, that is not much. A hairline font with rounded corners on 2048 UPM, inside a variable font, will jump awfully while dragging the slider to a darker weight. I was just working on such a font for an industrial customer that cares about quality, but the result is plain ugly. I need to go to 4096 to get acceptable (still not ideal) rounded strokes, but then some icons in the font won't fit in the 2^14 boundary. Using a UPM of 8192 can only work in narrow fonts.

The limit of 2^14 integers is definitely too small for high quality work. Either we need a bigger drawing board, or coordinates in fractional values.

(if this has already been taken care of in your proposals, hurray!)

[`sbix`] >64k expansion

Consume bit 2 of flags:

Bit 2: Use numGlyphs from loca table.

If bit 2 is on: for numGlyphs, use value derived from loca table instead of one from maxp table.

See #8 and #6.

VarFonts2.0 variation model

As referenced in #17:

struct GenerailzedRegion {
  Array15Of<BEOffset16To<Hill> hills;
};

struct Hill {
  uint8 version;
  Array8Of<BEOffset16To<Base>> bases;
  F2DOT14 scale; // only if version >= 1;
};

struct Base {
  uint8 version;
  F2DOT14 peakCoord;
  F2DOT14 startCoord; // only if version >= 1
  F2DOT14 endCoord; // only if version >= 1
  uint8 baseFuncs; // only if version >= 2
};

The Array15Of is intentional, meaning that the top bit of the uint16 count is reserved for future.
All offsets are relative; that is, they are based off of their own start address; As per #45.

struct Base {
uint8 version;
F2DOT14 peakCoord;
F2DOT14 startCoord; // only if version >= 1
F2DOT14 endCoord; // only if version >= 1
uint8 baseFuncs; // only if version >= 2
};

If startCoord is not available, it will be assumed to be the lesser of peakCoord and 0.
If endCoord is not available, it will be assumed to be the greater of peakCoord and 0.
If baseFuncs is not available, it will be assumed to be 0.

Interpolation algorithm

Interpolation algorithm for each base differs from VarFonts1.0, in many ways:

In VarFonts1.0, peakCoord cannot be zero (that is, peakCoord of zero has special meaning). Moreover, startCoord and endCoord must both be non-positive or both non-negative. None of these requirements and interpretations carry to VarFonts2.0.

The scalar value for a base is 1.0 at peakCoord.

If startCoord is the special value 0x8000 (-2.0), then the scalar value will be 1.0 for any coordinate less than peakCoord as well. Otherwise, the scalar will be 0.0 for coordinates less than startCoord, and will transition from 0.0 to 1.0 from startCoord to peakCoord according to the basis function specified in baseFuncs (default linear).

TODO hash out base funcs after #43 is filled in.

TODO: Add extrapolation.

The situation is symmetrical for endCoord.

TODO: baseFuncs.

[`OS/2`] Relax

Declare as optional, as it always has been in most implementations.

[`avar`2] Enable non-orthogonal axis distortions

Update: the aim of this proposal is just to enable non-orthogonal axes distortions. The higher-order interpolation will come by way of ItemVariationStore upgrades, via eg. #17

Update: Settled on the following format:
If version is 0x00020000, the format1-like struct is followed by the following fields:

struct avar2 {
  Offset32To<DeltaSetIndexMap> axisIdxMap;
  Offset32To<ItemVariationStore> varStore;
};

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.