mozilla / twemoji-colr Goto Github PK

Twemoji font in COLR/CPAL layered format

License: Other

JavaScript 92.46% Makefile 2.42% CSS 2.62% HTML 2.22% Python 0.27%

twemoji-colr's Introduction

twemoji-colr

Project to create a COLR/CPAL-based color OpenType font from the Twemoji collection of emoji images.

Note that the resulting font will only be useful on systems that support layered color TrueType fonts; this includes Windows 8.1 and later, as well as Mozilla Firefox and other Gecko-based applications running on any platform.

Systems that do not support such color fonts will show blank glyphs if they try to use this font.

Getting started

This project makes use of grunt-webfonts and an additional node.js script. Therefore, installation of Node.js (and its package manager npm) is a prerequisite. Grunt will be installed as a package dependency — no need to install it globally.

The necessary tools can be installed via npm:

# install dependencies from packages.json, including `grunt-webfonts`.
npm install

The build process also requires fontforge and the TTX script from the font-tools package to be installed, and assumes standard Perl and Python are available.

Both FontForge and font-tools can be installed via homebrew on OS X, or package managers on Linux:

# OS X
brew install fonttools fontforge

# Ubuntu, for example
sudo apt install fonttools fontforge python3-fontforge

Building the font

Once the necessary build tools are all in place, simply running

make

should build the color-emoji font build/Twemoji Mozilla.ttf from the source SVG files found in twe-svg.zip file and extras, overrides directories.

twemoji-colr's People

Contributors

Stargazers

Watchers

twemoji-colr's Issues

Repo description on GitHub still refers to EmojiOne

Hi all,

I noticed you are using twemoji graphics now, and the name of the repo has changed to reflect that, but the description of this repo at GitHub still mentions EmojiOne:

EmojiOne font in COLR/CPAL layered format

update to emoji 5.0

hey !
We have new emoji with emojis 5.0. it's possible to update ?
Regards

update to Twemoji v11.0

A CONTRIBUTE.md explaining the release process

Jonathan and I agreed that we should document the process of update the font, so others could take over in the future. I will document my part, however what Johnathan did (the actual, technical challenge parts) is a bit complex and involves cleaning up SVG layers with smart-enough logic. I will let him contribute that part later once I commit the file.

I hope I could get this done quickly before my vacation!

Add a license to the binary releases

Hi all,

I am interested in redistributing the binary font from this repo, by including it in a project I am working on.

But it struck me that there is no explicit association of any standard license, licensing terms, etc. to the binary releases.

I feel like to be legally prudent, I need to either direct people to compile the font themselves, or else ask that you kindly add a license of some kind to the binary release.

Thank you for considering.

(Info on the project:

I am writing a simple HTML page that tests browser support for all four OpenType color font formats. It will attempt to load fonts of each format as a locally installed font, and in @font-face to simulate usage as a web font. (I will compile and post results for all major browsers on as many platforms as I can get a machine to test on.)

Distributing the source of my test, along with the same fonts files I test with, makes the test 100% inspectable and reproducible.

As far as I know, this is the only publicly available, open-source COLR/CPAL font.)

Should version number reflect Unicode/Twemoji version?

With the last major release Twemoji bumped their version number from 2.7 all the way to 11.0 to align with the Unicode/Emoji version:

We've also changed the major version numbering of the Twemoji project to align with Unicode's versioning to better communicate the level of Unicode support included in the Twemoji library.

Is it worth doing something similar with this project? e.g. perhaps #37 should bump to 0.11.0.

OT: Build with national flags

This is not an issue that needs to be resolved, but I do not know why the release versions do not include one with the national flags.

Anyone successfully built a version of the font with the national flags and could share with me?
I am having version compatibility issues on my PC and could not build it.
Please, I need one for testing. Thank you.

Can’t get the font to work on Linux

With a simple test like this I always get the emoji shown in a fallback font in Firefox on Linux:

<html>
  <head>
    <meta charset="utf-8"/>
  </head>
  <body>
    <p style="font-family: EmojiOne Mozilla;">🐻</p>
  </body>
</html>

I can’t debug this right now, but I suspect this is because the default glyphs are all blank and FontConfig (in its eternal wisdom) ignores blank glyphs in the font and will decide that the font does not support these characters (unless they are characters it thinks can be blank).

Test page: the result of Inspect Layers may be incomplete

For some characters, Inspect Layers only shows a subset of the layers actually used to make up the complete layered glyph. (E.g. lots of the smiley faces.) This is because it infers the list of layer from the glyph names; but this may be incomplete because where possible, the layerize script tries to share components between multiple characters. So when a set of characters have lots of components in common, those components will only appear in the Inspect Layers output for the first character where they were encountered, as that determines their glyph names.

To fix this, we should make Inspect Layers work from the actual list of layers for each colored glyph, rather than trying to work back from the collection of named layer glyphs which may be shared.

Update to 14.0.0

Upstream 14.0.0 relase which supports emojis from Unicode 14

Make our packages.json

After #1 is fixed.

I would also like to install grunt-cli and grunt as dependencies, instead of asking user to install it globally (which avoids hitting old version of it).

Some national flags result missing layers

This issue does not block v0.2.2 from shipping because it's always here.

U+1f1e6 U+1f1eb, 🇦🇫, reference: true, rendering: true, webfont: true, mismatch: 4.62%, retested: false, layers: 9flag for Afghanistan. tags: afghanistan, flag, other.
U+1f1fb U+1f1fa, 🇻🇺, reference: true, rendering: true, webfont: true, mismatch: 2.66%, retested: true, layers: 6flag for Vanuatu. tags: flag, other, vanuatu.

Redundant word "Version" in TTF version string

I notice that the .ttf file ends up with a version string (name id 5) that reads:

<namerecord nameID="5" platformID="3" platEncID="1" langID="0x409">
  Version Version 0.3.1
</namerecord>

I guess this is because https://github.com/mozilla/twemoji-colr/blob/master/Gruntfile.js#L22 prefixes the word "Version" to the number, but then the webfont generator must be automatically doing the same thing.

So presumably just removing this from the gruntfile should fix it. (Untested.)

Emoji is solid black in latest Firefox

I'm in the latest version of Firefox. The emoji are displaying as solid black.
The emoji is not displaying at all in Chrome, even though this format is now supported by Chrome.

npm install the original `grunt-webfont`

After sapegin/grunt-webfont#335 is merged.

Update to 13.1.0

Upstream 13.1.0 release

Create a new fixed-up release ttf

Release 0.3.1 has issues in the .ttf caused by different tags in the Twemoji source files compared to EmojiOne, which have been solved by commit 98ff7b2
It would serve community well to create a new release of twemoji-colr (e.g. 0.3.2) making the fixed-up ttf available to projects that do not have an easy way to run the layerize script themselves (e.g. on Windows).

Test the built result?

How do you design a test script for the resulting font? The only thing I could think of currently would be:

Fire up Firefox in an automatic faction (via SlimerJS)
Load a test web page that would load the built font as web font
Go through the code points, paint each of the glyth on a <canvas>
Ensure they look differently than the system default (implies EmojiOne is encoded on the right code point and is rendered correctly)

I am not sure about (4) is a worthy assertion to set this up -- or if what we could improve from there.

Ubuntu Getting Started instructions miss out python-fontforge package

Trying to follow the current instructions results in

Traceback (most recent call last):
  File "fixDirection.py", line 1, in <module>
    import fontforge
ImportError: No module named fontforge

after make.
I had to (discover and) install the python-fontforge package from apt to be able to build successfully.

Update to 13.0

Upstream 13.0 release

Update to Twemoji v12

This isn't imminent, but filing for tracking.

Wait for v12 to bed down a little more first (12.0.1 just released adding missing hair components)
Wait for emojibase to update: milesj/emojibase#20

Important metadata missing in ttf file

Important metadata is missing in the ttf file (v0.6.0). The most important is license.
It's also important to add the corresponding Twemoji version (could be in description field or appended in version field).

For comparison, TwitterColorEmoji-SVGinOT.ttf (v13.1.0, Linux) has at least these:
Version: 13.1.0 20210621
Copyright: Copyright 2021 Brad Erickson CC-BY-4.0 Copyright 2021 Twitter, Inc. CC-BY-4.0
Description: A SVGinOT color emoji font using the Twitter Emoji for Everyone set: https://github.com/eosrei/twemoji-color-font
Manufacturer: 13rac1
Designer: Twitter, Inc.
License: Creative Commons Attribution 4.0 International

So, I assume it should be similar, except having Mozilla as manufacturer and linking to this repo. And appropriate copyright for whatever version it's using.

Should we care that some of the Twemoji filenames don't match their actual sequence

See twitter/twemoji#272.
They are missing FE0F: VS16.
This affects keycap sequences and the "eye in speech bubble".
Should we give them the proper sequences during font creation? Easy enough to do during processFile in layerize.js.
Though for some reason they still seem able to be used for the actual sequence in pages (but the keycaps require explicit font-family: "Twemoji Mozilla"; for some reason).

cc @jfkthame

Add a set of 26 "boxed" A-Z glyphs for the individual Regional Indicator letters

They're not present as individual images in the upstream collection, but it's useful to have them so that unrecognized RI sequences, or individual characters (e.g. during editing) can be seen properly.

The EmojiOne-SVGinOT font did this, creating letters on green background tiles; other implementations have things like a letter in a dotted box. I'd prefer the latter, as the green-background letters in eosrei/emojione-color-font look too much like they're intended for standalone use in their own right, which isn't appropriate.

v0.6.0 release has outdated binary uploaded

~/Downloads
❯ sha256sum .\TwemojiMozilla(2).ttf
860b69e096e5805015cf5b5d64e4ece06c5b987dc05da1f97835c79d9cc79b10  .\TwemojiMozilla(2).ttf

~/Downloads
❯ sha256sum.exe .\TwemojiMozilla(3).ttf
860b69e096e5805015cf5b5d64e4ece06c5b987dc05da1f97835c79d9cc79b10  .\TwemojiMozilla(3).ttf

I noticed it was the same size as the 0.5.1 binary, and that most of the emoji 13 glyphs were missing, so I checked and they're the same file.

consider what font family name we should use

I've been building the font with the name "EmojiOne" for now, but I'm not sure that's really a good idea. Given that this will not be a "perfect" conversion of the original EmojiOne graphics -- because there are a few features that are hard to represent precisely in a color-layer font -- and that it's not officially endorsed by the EmojiOne project, it might be better to use a name that clearly identifies our font as a Mozilla derivative.

Note that for our intended use case (bundling with Firefox), the font name will not normally be directly exposed to users, so in that sense it doesn't much matter what we call it. But of course it's possible people will take the font and install it separately for use with other applications, too. That's fine ... but in scenarios like that, I think it's desirable that we have a more distinctive name than plain "EmojiOne". We should leave that name for the EmojiOne project to use themselves for any font products they release, and avoid a potential name clash with our custom derivative.

Some possibilities: "EmojiOne-Moz"? "MozEmojiOne"? "FxEmojiOne"? Something else....?

Run tests from command line and save results to a folder?

The test page/script is awesome, but I've been thinking it might also be nice (and less resource-hungry for the browser) to have the option of running tests in a "batch" mode from the command line, generating a report similar to the current test page (but as a static file), with the test images and difference saved to an associated folder as a collection of PNGs.

This would allow us to have a "make check" command that runs the tests and reports a summary of the stats, and the results could then be reviewed in the browser at leisure. It might also make it easier to explore alternative image-diffing tools as part of the test script -- it would be nice if we could find a tool that can be "tuned" to report fewer failures where the only discrepancy is along the fringes of areas, while still detecting "real" differences. (E.g. maybe ImageMagick's compare tool could be useful?)

The test page is not working for me

Since I updated to the new EmojiOne release, the test script no longer works (at least for me; maybe it's a local configuration issue?) I don't see any errors in the DevTools console, but clicking Test just doesn't seem to do anything.

@timdream could you look into this, please? Currently, I haven't been able to re-run the test script with the new version of the graphics, so I have only done a superficial visual check that things look OK-ish...

Switch to Emoji Two

I believe you're aware that the latest versions of Emoji One are no longer available under a license that allows free modification and distribution.

Please switch to Emoji Two https://github.com/EmojiTwo/emojitwo
It doesn't have an official release yet, but I think a git snapshot would work.

Twemoji 15 Update

Hey there,

It'd be great to update to Twemoji 15 using jdecked's (original designer) fork, considering the original repo has been abandoned due to the "instability" over at Twitter.

Thanks,
Elliott

CSS font-palette documentation

I’m trying to use this font in conjunction with the CSS font-palette property (currently available in Safari Technology Preview). When defining override-colors in CSS there seem to be loads of layers in this font. Would it be possible to document which layers/Color’s correspond to an override-color number. E.g. if I set override-colors: 4 red; which part of the font would become red?

Provide updated releases on GitHub

Please provide updated binary releases on GitHub. They are still stuck at 0.3.2, and e.g. https://hg.mozilla.org/mozilla-central/rev/9bdb1897c29a indicates there is at least a 0.4.0 -- providing binary releases here will allow other projects than Firefox to follow suit.

Flag of Martinique not showing snake eyes

The original svg file has small blue eyes on the snakes. This is not represented in the font.

Build doesn’t work due to 'unzip' dependency

The make is failing in the unzip dependency, which is not maintained for like 9 years.

% make
node layerize.js twe-svg.zip overrides extras build Twemoji\ Mozilla
fs.js:47
} = primordials;
    ^

ReferenceError: primordials is not defined
    at fs.js:47:5
    at req_ (node_modules/natives/index.js:143:24)
    at Object.req [as require] (node_modules/natives/index.js:55:10)
    at Object.<anonymous> (node_modules/fstream/node_modules/graceful-fs/fs.js:1:37)
    at Module._compile (node:internal/modules/cjs/loader:1103:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1155:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
make: *** [build/codepoints.js] Error 1

The dep tree branch:

├─┬ [email protected]
│ ├─┬ [email protected]
│ │ ├─┬ [email protected]
│ │ │ └── [email protected]

CODE_OF_CONDUCT.md file missing

As of January 1 2019, Mozilla requires that all GitHub projects include this CODE_OF_CONDUCT.md file in the project root. The file has two parts:

Required Text - All text under the headings Community Participation Guidelines and How to Report, are required, and should not be altered.
Optional Text - The Project Specific Etiquette heading provides a space to speak more specifically about ways people can work effectively and inclusively together. Some examples of those can be found on the Firefox Debugger project, and Common Voice. (The optional part is commented out in the raw template file, and will not be visible until you modify and uncomment that part.)

If you have any questions about this file, or Code of Conduct policies and procedures, please see Mozilla-GitHub-Standards or email [email protected].

(Message COC001)

Not showing numbers on input like 1 , 2, 3 using this font family.

Using this font-family showing flag properly on chorme but not showing any numbers in text-area field.

UN flag doesn't match up to the source SVG

I've found through testing that the UN flag emoji (U+1F1FA, U+1F1F3) is slightly messed up in the middle, as you can see here:

The continents are larger and overlap each other. It seems to happen after the layerize script, as each layer SVG is correct. Maybe it has to do with grunt-webfont? 🤔

Should MINIMAL_REPORT: true retain failed reports?

At the moment setting MINIMAL_REPORT to true discards all results (which is handy, since otherwise my 16GB Linux system grinds to a halt while running the tests in Nightly) however the resulting report is limited in use since you can't see the failure comparisons. Perhaps only passing reports could be thrown away when this value is set? (Or else some new variable is used to indicate such behaviour)

A local change implementing such has been rather helpful.

Misencoded emoji in EmojiOne 2.2.1

EmojiOne 2.2.1 puts a few emoji at the wrong code points; see joypixels/emojione#258. This is fixed in EmojiOne 2.2.2.

Glyphs do not render on Firefox 66 or Chrome 74 on MacOS 10.14.4

Trying both the latest release and a self-built copy of the TTF, the glyphs seem to render as blanks on FF & Chrome on macOS. Safari 12.1 however works.

Poking inside the TTF using Glyph.app shows that generated glyphs seem to have no geometry on the regular base layer; only on the colour layers. Defining some geometry on the base layer seems to fix it (e.g. a bounding box, or a copy of the biggest colour layer).

I'm a bit unclear on what the right fix is for this, or how it has ended up breaking so badly. Is it to calculate a bounding box and put it in the regular layer? or something else?

The resulting font is not determinstic

Every make generates different bits. Not sure if we want to fix it.

Update to the 2016-05-31 release of EmojiOne

There's a new (v2.2) release of the EmojiOne set, so we should update the source archive we're using here (and re-test everything, obviously).

Is there an SVG-in-OpenType version?

Is this font available in the SVG-in-OpenType format?

Emojis

`npm install` failing

It seems like one of the prebuilt dependencies has been taken down, and I can't get this to compile locally. Am I doing something wrong?

> [email protected] install /home/tiffany/git/twemoji-colr/node_modules/canvas-prebuilt
> node-pre-gyp install

node-pre-gyp WARN Using request for node-pre-gyp https download
node-pre-gyp ERR! install error
node-pre-gyp ERR! stack Error: 404 status code downloading tarball https://github.com/chearon/node-canvas-prebuilt/releases/download/v1.6.11/canvas-prebuilt-v1.6.11-node-v72-linux-x64.tar.gz
node-pre-gyp ERR! stack     at Request.<anonymous> (/home/tiffany/git/twemoji-colr/node_modules/node-pre-gyp/lib/install.js:137:27)
node-pre-gyp ERR! stack     at Request.emit (events.js:215:7)
node-pre-gyp ERR! stack     at Request.onRequestResponse (/home/tiffany/git/twemoji-colr/node_modules/request/request.js:1066:10)
node-pre-gyp ERR! stack     at ClientRequest.emit (events.js:210:5)
node-pre-gyp ERR! stack     at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:583:27)
node-pre-gyp ERR! stack     at HTTPParser.parserOnHeadersComplete (_http_common.js:115:17)
node-pre-gyp ERR! stack     at TLSSocket.socketOnData (_http_client.js:456:22)
node-pre-gyp ERR! stack     at TLSSocket.emit (events.js:210:5)
node-pre-gyp ERR! stack     at addChunk (_stream_readable.js:308:12)
node-pre-gyp ERR! stack     at readableAddChunk (_stream_readable.js:289:11)
node-pre-gyp ERR! System Linux 4.4.0-18362-Microsoft
node-pre-gyp ERR! command "/usr/bin/node" "/home/tiffany/git/twemoji-colr/node_modules/.bin/node-pre-gyp" "install"
node-pre-gyp ERR! cwd /home/tiffany/git/twemoji-colr/node_modules/canvas-prebuilt
node-pre-gyp ERR! node -v v12.13.0
node-pre-gyp ERR! node-pre-gyp -v v0.10.3
node-pre-gyp ERR! not ok
404 status code downloading tarball https://github.com/chearon/node-canvas-prebuilt/releases/download/v1.6.11/canvas-prebuilt-v1.6.11-node-v72-linux-x64.tar.gz

Load eosrei/emojione-color-font as submodule?

It looks odd to me that we are checking in a zip containing all the SVGs. Would it be better if we track the version of the upstream asset by adding the upstream repo as a submodule?

Generate COLRv1 version to use this font in more situation

Firefox has supported COLRv1 since 107, Chrome supported since 98, FreeType, Harfbuzz and Pango/Cairo also supported COLRv1.
But COLRv0 is only supported by a little software. We can't use this font in GTK/Qt application.
So I hope this font can has COLRv1 version to be used more situation.

Ref:
https://gitlab.gnome.org/GNOME/pango/-/issues/752
https://bugreports.qt.io/browse/QTBUG-114324

[Test suite] ReferenceError: options is not defined

With newer versions of resemblejs, when you try to run the test suite, it throws this error:

ReferenceError: options is not defined
compare@http://twemoji-colr.herokuapp.com/node_modules/resemblejs/resemble.js:636:5
wrapper@http://twemoji-colr.herokuapp.com/node_modules/resemblejs/resemble.js:779:7
onComplete@http://twemoji-colr.herokuapp.com/node_modules/resemblejs/resemble.js:782:6
imageCompare/</<@http://twemoji-colr.herokuapp.com/index.js:314:11
imageCompare/<@http://twemoji-colr.herokuapp.com/index.js:312:16

It seems this has to do with the new resemble.compare API in version 2.6.0. We might need to rewrite the comparison script to work with the new version.