[otf2otc] Support `glyf` table merging about afdko HOT 26 CLOSED

Have you tried using the -t command-line option, such as:

otf2otc -t 'glyf'=0

to forcibly specify that the glyf table is to be shared? That is why this command-line option exists.

from afdko.

@kenlunde Nope, I'd like to MERGE glyf tables instead of picking one of them.
Consider two ttfs, a.ttf contains glyph A and B, b.ttf, contains glyph A' and B, the result ttc should contain a new glyf table containing glyphs A, A' and B, and recalculated cmap, GSUB and GPOS tables maintaining the unicode-glyph mappings, as well as Opentype features.
Pretty like the "Gemerate TTC" in Fontforge with "merge tables across fonts" turned on.

from afdko.

@be5invis It sounds like you’re looking for something like what https://github.com/behdad/fonttools/blob/master/Lib/fontTools/merge.py is aiming to do.

from afdko.

@moyogo I think it's different from what I want: I'd like a merging program merges several TTFs into a TTC, where the source TTFs covers similar Unicode ranges.

from afdko.

@moyogo: Indeed. The otf2otc script operates on whole sfnt tables, and simply manages the sharing of tables among the font instances. What @be5invis needs is something that operates further upstream, similar to the AFDKO mergeFonts tool.

from afdko.

@kenlunde no, the thing I need is an enhanged version of otf2otc which can merge glyf table across the source ttfs to reduce the final file size. It is pretty old-school, since this glyf merging are used in early East Asian fonts in Windows, providing both proportional and monospaced variant in one single ttc file.

from afdko.

@be5invis what about merging the glyf tables to two ttf in a singe ttf, then building a ttc with the glyf table of that ttf?

from afdko.

@be5invis: I understand what you want to do, and I am merely stating that otf2otc is the wrong tool for the job. Keep in mind that a collection is merely a container format, and that the individual fonts that comprise it need to be able to stand alone if the otc2otf script is used to separate them into individual font resources. The combining of the glyf tables need to be done further upstream, before running otf2otc. In other words, you need to feed to otf2otc fully-functional font resources.

from afdko.

@moyogo like this:

The upper rows are two source ttfs showing the cmap and glyf table they contain, and the lower row is how these tables should be in the final ttc.

from afdko.

@kenlunde So are there any proper tools is provided in FDK to do this? The input ttfs may be hinted and may contain special metadata (like a manually specified xAvgCharWidth).

from afdko.

The AFDKO tools that operate on glyf tables are limited. I am not sure whether mergeFonts can operate on that table, but in addition to merging multiple glyf tables, which requires careful mapping in terms of which glyphs to share, the cmap table also requires adjustment.

I suggest that you start by building a superset glyf table, which is then used in two or more TTFs whose cmap tables reference a different subset of the glyphs in this superset glyf table. Once you have built such TTFs, you can then feed them to otf2otc to produce a well-formed and functioning-as-expected TTC.

from afdko.

However such table merging and adjustmens are ONLY used when building a glyf-merged TTC. Maintaining a "merged" glyf is not a simple work for font authors.

Authors may have to maintain these tables if the glyf-merging is not provided in the "ttc merger":

• a merged glyf
• cmap, hmtx, hdmx, vmtx, GSUB, GPOS where all glyph indexes are "rearranged"

from afdko.

Although it is CFF-based, that is precisely what I did for the Source Han Sans project. The CFF table includes all glyphs (a superset), and four language-specific cmap tables reference a different subset of the glyphs in the CFF table. The very last step is to feed the individual (language-specific) OTFs to otf2otc to produce an OTC.

from afdko.

CFF contains all outline and metrics data of a Postscript-outlined font, howerer for TTF-outlined fonts, these data are separated in several tables, which makes maintaince hard.

from afdko.

Have you considered building CFF-based fonts instead?

In any case, the processes that you have described are non-trivial, and I am sure that @readroberts will agree that otf2otc is not the right tool for this.

from afdko.

@kenlunde I have to provide Truetype-outlined TTCs, because they are used for screen display and terminal typeface on Windows. They contain a lot of gridfit instructions (takes over 50% of the file size) and something special in the low level to ensure they are displayed properly.

from afdko.

I understand the urge to have function to merge font tables that are in two or more fonts, but that is a s completely different function than otf2otc. The point of a TTC file is that it is, at a high level, simply several different fonts stuck together in a single file, with the bonus that you only need one copy of tables that are identical. otf2otc identifies identical or non-identical OpenType tables without any understanding of what is in the data, and sticks them together. It is by design simple, stupid, and fast. The additional functionality you want is fundamentally different: take several different fonts, and merge their data into a single font. It does not make sense to me to have the hugely complex function of merging two font files into one pushed into the subsequent step of building an OTC; the former should be a separate program.

Such programs do exist. The only OpenSource program (that I know of) which does merge several fonts into one font is indeed mergeFonts, referenced earlier in this thread. I would never attempt to develop such a program for the AFDKO without dire need and serious funding, and would in any case then put my effort into extending 'mergeFonts' instead.

from afdko.

@readroberts so how to "prepare" these ttfs with glyf merged using mergeFonts?

from afdko.

I'm sorry to say that I can't help you there. I have never used it, and am aware of its existence only because of comments on the opentype mailing list. I suggest you check out a copy of the program and look at its documentation. If that is not clear enough, then perhaps you could ask for advice from developers for the repository. I just looked at it now, at:
https://github.com/behdad/fonttools/blob/master/Lib/fontTools/merge.py
and see that it is actually just a module for the 'ttx' program, a well known Python-based program for dumping OpenType fonts to XML, and compiling them back from XML files, so the repository is actually 'https://github.com/behdad/fonttools', which is where you would ask questions.

from afdko.

@be5invis Read is correct. The workflow you need is the following. Let's assume you have 4 source .ttf fonts: fonts A1.ttf, A2.ttf, A3.ttf A4.ttf.

1. Use the ttx -t tool from the fontTools package on each of the A1.ttf-A4.ttf fonts, to export into .ttx XML files those SFNT tables from these fonts which do NOT involve any glyph ID references and that you intend to keep identical in your final .ttc. For example, you might want to do ttx -t name -t head -t 'OS/2' -t hhea A1.ttf etc. You'll end up with four .ttx files: A1.ttx-A4.ttx. Keep those.
2. Use pyftmerge (the tool from fontTools using merge.py) to merge the fonts A1.ttf-A4.ttf into one large B1.ttf font that will automatically merge all SFNT tables of the single fonts, including the glyf table (and all other glyph-dependent tables like hmtx etc.), and will automatically rename all the glyphs by appending suffixes, and will renumber the glyph ids in all the SFNT tables.
3. Use the pyftsubset tool (fontTools using subset.py) to take font B1.ttf and create four .ttf fonts ("fonts C1.ttf-C4.ttf") where each will contain the glyph complement of each of the destination subfonts (for each Cx.ttf output font, you need to supply pyftsubset with a list of glyph names, using the NEW glyph naming scheme that was created by pyftmerge). That will keep the glyph names devised by pyftmerge in each of the subsetted Cx.ttf fonts, but will of course "lower" the glyph IDs so each .ttf will have glyph IDs starting from 0.
4. Use the ttx -t tool on each of the fonts C1.ttf-C4.ttf to export those SFNT tables which involve the glyph IDs and that you want to keep separate in each subfont into XML files. Here, you might use ttx -t cmap -t GSUB -t GPOS -t post AC.ttf. This will result in files C1.ttx-C4.ttx. Move these .ttx files into a new folder.
5. Take the resulting files C1.ttx-C4.ttx and on each one use the ttx -m tool to merge the font B1.ttf into them (ttx -m B1.ttf C1.ttx, ttx -m B1.ttf C2.ttx etc.). Since ttx operates on glyph names, it will take the font B1.ttf and on each run will write into it SFNT tables that are found in the C1.ttx-C4.ttx files. It will use the glyph names to match the glyphs, and will renumber the glyph IDs in those tables to match the glyph IDs of the B1.ttf font. You will end up with fonts C1.ttf-C4.ttf which will have the same glyf table (and whatever else tables you did not export into .ttx in step 4), but each will have a different set of subsetted tables that you did include in step 4. Move those C1.ttf-C4.ttf files into a new folder.
6. Use the ttx -m tool to merge these new C1.ttf-C4.ttf fonts with the A1.ttx-A4.ttx files you created in step 1: ttx -m C1.ttf A1.ttx etc. You'll end up with new fonts A1.ttf-A4.ttf which have the original non-glyph-dependent tables, unique "subsetted" tables that were created by pyftsubset, and identical combined tables which you left intact.
7. Finally, run otf2otc on the new A1.ttf-A4.ttf font files to create a final .ttc.

I hope it makes sense to you :)

Best,
Adam

from afdko.

Ps. Principally, you could use mergeFonts instead of pyftmerge if you’re working with CFF-based .otf fonts, and you may run into various troubles. I have NOT tested the workflow that I just described myself, but it should work in principle. But the above is conceptually the process that I would employ if I were to achieve the thing you’re aiming for. Please feel free to report your findings, but I want to stress that I won’t be able to lend any specific assistance if you run into troubles.

One tool that may also be useful is sfntedit from AFDKO, which works on .ttf/.otf files and dumps and merges SFNT tables. Keep in mind that sfntedit treats SFNT tables as binary blobs, so you need to be aware of GID issues (it won’t renumber anything). ttx on the other hand operates on glyph names, so it will match glyph references using their names, and will renumber the GID references in all kinds of tables.

from afdko.

@twardoch Does pyftmerge handle gridfits properly? Two glyphs with identical outline BUT different gridfit should be treated as different.
Also, can it handle TTF references correctly?

from afdko.

@be5invis pyftsubset handles TTF components correctly (it will employ a "greedy" tactic, i.e. if there are composite glyphs that you specify in your target subset, it will also include the necessary outline glyphs that are references by the composites.

As far as pyftmerge goes, I have not tested it much, so you’ll need to experiment yourself.

Finally, in addition to pyftmerge, FontLab apps (FontLab Studio 5 and TransType 4) offer font merging. FontLab Studio 5 will let you control what to do with glyphs that have the same names in different fonts (omit, overwrite or append suffix). But it won’t do much smart stuff with other SFNT tables.

Generally, no magical font merging tool exists that would be supersmart. So the merging part is the toughest. But once you have a big merged font, the pyftsubset+ttx+otf2otc workflow that I described actually does work well. I've tested it.

In fact, my tool https://github.com/twardoch/fonttools-utils/tree/master/pyftfeatfreeze can be used for such partial purpose. It remaps the cmap table by hardcoding various GSUB features, and allows simple renaming in the name table. You can generate a number of such fonts with it which will differ only in the cmap and name tables, and then merge the result using otf2otc, and it will work quite nicely.

But yeah, merging is tedious, and you’ll need to figure out things yourself.

from afdko.

Also: if you need better control of how the font merging process happens, you may need to employ some more operations, e.g. use pyftsubset once more earlier, before using pyftmerge to remove the duplicate glyphs from the fonts that you’re going to merge. Also, you may want to rename glyphs in the input fonts yourself accordingly, using the snippet fonttools/fonttools#149 before you do the merging.

As I said, merging is complicated :)

from afdko.

Conclusion: otf2otc is for merging fonts into font collections and not for merging tables (glyf or otherwise).

from afdko.

Sorry for replying to such an old thread.

The workflow you need is the following. Let's assume you have 4 source .ttf fonts: fonts A1.ttf, A2.ttf, A3.ttf A4.ttf.

@twardoch, in this example workflow, how would you propose recovering the original A1.ttf-A4.ttf files given a .ttc with merged tables?

from afdko.