[Linux] Inconsistently parsed fontconfig causes font selection problems about emojione-color-font HOT 24 OPEN

Possible "solution": Use the DejaVu Sans number glyphs.

(quotes intentional)
This doesn't actually fix the problem, but it would solve, aka mask, the issue for the vast majority of people.

from emojione-color-font.

A good example of what the disabled glyphs cause by @kgizdov, further details in #46 :

from emojione-color-font.

Following your comment in #46 about ligatures. As far as I understand how these work, you need two characters to form one. In this case you need a number followed by the emoji glyph. So what I still don't understand is why would this make a number appear as the emoji glyph even if the application is loading the wrong font. If the font has the number and the emoji glyph, it should be also able to provide only the number when requested, no? I'm not an expert, but couldn't this be caused by the font always returning the glyph, even when only the number is requested?

from emojione-color-font.

Ah, I'm not explaining it well enough. :)

Terminology (Rough):

• Glyph = The vector shape used to draw a character on screen. There are b&w and SVG color glyphs in this font.
• Code point = The Unicode ID number of a specific character or symbol
• Character = Generally, the representation of a code point drawn using the data referenced in a glyph.
• Ligature = A combination of code points replaced by a single glyph.

If the font has the number and the emoji glyph, it should be also able to provide only the number when requested, no?

Originally, I had to put something there at the 0x30 through 0x39 unicode code points, so I made the keypad numbers into "emoji-style" numbers. The new version of EmojiOne now includes "emoji-style" numbers at those code points so, my custom SVGs are no longer needed. Here's one (haha) from the removal commit listed above: https://github.com/eosrei/emojione-color-font/blob/e1a34da33493ee2966012cc6467ad519dfc518d3/assets/emojione-svg/disabled/0031.svg

There must be a glyph at 0x30 and 0x20e3 in the font providing the ligature for the 0x30-0x20e3 ligature to work. When EmojiOne Color is selected incorrectly as default by MPV, Steam, Evolution, etc then they load the EmojiOne 0x30 instead of the DejaVu 0x30. These, AFAIK, are bugs in all three programs.

Solutions:

• Create a Fontconfig which works everywhere. Current fontconfig: https://github.com/eosrei/emojione-color-font/blob/e1a34da33493ee2966012cc6467ad519dfc518d3/linux/fontconfig/56-emojione-color.conf How to fix?

Workarounds:

• Export the DejaVu Sans glyphs and use those for 0x23, 0x2a, and 0x30-0x39. I'd rather not do this.

from emojione-color-font.

I think you've explained it very well, but I also think you're not doing what the UTF standard asks. UTF implements ASCII and 0x30 through 0x39 (and other ASCII codes) should be represented the same in the wider 0x0030 - 0x0039 by any font following the standard. So having the glyphs for emoji numbers in the places for normal numbers is wrong. The glyph (as the standard requires http://www.fileformat.info/info/emoji/zero/index.htm), for emoji number 0 for example, should only be returned if codes 0x0030 and 0x20e3 are requested together in succession. Otherwise 0x0030 should be just the normal number 0 glyph. Why not draw your own numbers? You don't have to import DejaVu glyphs. But having the number glyphs is obviously needed to actually be following the spec.

from emojione-color-font.

I was using 0x30 as the short form of 0x0030 since they are the same number and the extra zeros are redundant. Nothing about ASCII intended.

for emoji number 0 for example, should only be returned if codes 0x0030 and 0x20e3 are requested together in succession

That's exactly how the font works.

So having the glyphs for emoji numbers in the places for normal numbers is wrong.

Yes, the font doesn't do that.

Why not draw your own numbers?

EmojiOne already has it's own representation of regular numbers: https://github.com/Ranks/emojione/blob/8e4691229ed434834fb2fa97888d596986d3c636/assets/svg/0030.svg and I've disabled them: https://github.com/eosrei/emojione-color-font/tree/master/assets/emojione-svg/disabled which is why we are discussing this.

But having the number glyphs is obviously needed to actually be following the spec.

Yes, but I cannot, because then I have to re-open all the issues about MPV, Steam, and Evolution. Plus, I expect I'd get many more. I've described the ideal solution which I don't know how to solve and potential workaround which I'd rather avoid. Any help on a Fontconfig which will work everywhere is greatly appreciated. Thanks.

from emojione-color-font.

I also think you're not doing what the UTF standard asks.

Please dissemble V1.2 of the font and check. As far as I know, I am following both the UTF and OpenType spec, except I have disabled the emoji numbers aka keycap numbers due to all listed issues.

from emojione-color-font.

OK, I went and opened up the fonts to see what's going on.
No issue with 0x20e3:

But as suspected 0x0030 is strange in v1.1:

In fact many of the symbols which are plain text ASCII have emoji glyphs. For contrast here are some screenshots from Google Noto Emoji, Bitstream Vera, DejaVu

Noto:

Bitstream Vera:

DejaVu:

Having started on this, I also noticed the following issue in Chrome:

Google's Noto answer to this is like that:

Now I understand that the issue arises from the fact that the UTF codes for these correspond to the fallback outline emoji and in Firefox and other apps that use the proper colour emoji this is not a problem. But I believe, this is exactly the point. Now, I do not want to tell you how to make your font. You seem to have done a pretty good job as it is and I like it very much. However, I still think that the standard requires the ASCII codes to stand for ASCII type characters with ASCII type glyphs.

Thus I am of the opinion the proper way to handle this is the way Google's Noto does it. The outline Noto Emoji font has the ASCII glyphs for the characters in question and the Noto Color Emoji has the emoji type once. To add to this, I am not sure how the Noto Color emoji font works exactly, as I could not open it neither in FontForge nor in BirdFont. This is probably where your expertise fills in the gaps to have the outline and color emoji fonts work as the user expected.

However you decide to solve this, I just want to say - thank you very much for the font, it is the default for me and I am overall very satisfied with it. 😜

from emojione-color-font.

😄 These are not emoji:

They are the regular glyphs for numbers in EmojiOne (the graphic set). A stylistic representation of a number does not make it "emoji". Here are some examples of other fonts with far more interesting number representations.

Extra Sprinkles: http://www.dafont.com/extrasprinkles.font

Racing Numbers: http://www.dafont.com/racingnumbers.font

Solarium: http://www.dafont.com/solarium.font

There are a hundred thousand more examples just on on dafont.com. The number glyphs used in V1.1 of this font are correct.

Thus I am of the opinion the proper way to handle this is the way Google's Noto does it.

I argue it's a workaround, not a proper solution, but that's why I have it listed above.

Noto Color emoji font ... I could not open it neither in FontForge nor in BirdFont.

It doesn't follow the OpenType spec. It only has the CBLC-CBDT color glyphs. It doesn't contain any regular b&w glyphs, so Fontforge can't do anything with it.

from emojione-color-font.

Recently, I have noticed that if I have the font installed, Evince will not show greek characters (for math equations) correctly. The weird thing is, if I regenerate the PDFs from LaTeX source on my machine, it all works fine, but if it is generated elsewhere the characters are then missing when viewed on my machine. I have tested with PDF generated in LaTeX on Ubuntu 14.04 without EmojiOne installed and on my Arch Linux with EmojiOne installed. Not sure if that's Evince's fault or some font-config issue. If I uninstall EmojiOne, then I can see the characters but they look wierd - as if they are rendered with the wrong font face.

EDIT: It's definitely not Evince. I confirmed the same issue with other viewers.

from emojione-color-font.

Can you check Character Map? If you right click on a symbol it will tell you which font the gylph is coming from.

from emojione-color-font.

When in Character Map and having selected EmojiOne Color as font, the characters in question appear as DejaVu Sans.

from emojione-color-font.

And otherwise? That's what it should be.

from emojione-color-font.

It's definitely not Evince. I confirmed the same issue with other viewers.

Are they using a different backend library or the same? What about PDF.js in Firefox?

from emojione-color-font.

I think the proper font should be Arial - Italic to be exact. I'll have to double check PDF.js a bit later and come back to you.

from emojione-color-font.

If I uninstall EmojiOne, then I can see the characters but they look wierd - as if they are rendered with the wrong font face.

Arial isn't installed on my machine for example. You need to embed the fonts in the PDF.

from emojione-color-font.

So I tried both Chrome and Firefox. I also inspected the fonts in the document. Here's what I see.

• None of the fonts are embedded. They are listed as - Times New Roman, Helvetica, Courier New, ZapfDingbats, Symbols (not sure which this is exactly).
• They are substituted by NumbusRoman from NimbusRoman.otf and Bitstream Vera from Vera.ttf.
• Chrome and Firefox render the chars correctly
• Firefox uses Bitstream Vera for the whole document
• Chrome uses Roboto and Noto

And here's the important bit:

• if I remove EmojiOne the substitution fonts in Evince become - Numbus for Times New Roman, Liberation Mono, DejaVu Sans for ZapfDingbats and Symbols.
• The change is Courier from Bitstream Vera to Liberation Mono and Symbol and ZapfDingbats from Bitstream Vera to DejaVu Sans

I hope that helps.

from emojione-color-font.

This is yet-another example of why this overall issue needs to exist.

The solution in your case is to embed the fonts in the PDF. It's the only way to make PDFs look consistent everywhere. Just like web development, you cannot trust the client! Here is Adobe's description of how/why: https://helpx.adobe.com/acrobat/using/pdf-fonts.html

I have no idea about a solution for the overall issue, but that is why I have this issue labeled "Help Wanted". Font configuration issues on all three OS have now take far more time than the actual font itself. I had no idea what I was getting into when I started this project. Haha! So many bugs in everything.

from emojione-color-font.

You will probably also like the fact that if I install the xorg-fonts, which some people say will help with this, the substitution font becomes EmojiOne. This of course is even worse... I'm not sure what to do. Is there a way to specify order of font fallback per application or something like that. The Arch Wiki seems to be kind of unclear on that front.

from emojione-color-font.

Is there a way to specify order of font fallback per application or something like that.

Yes, you can, but you'd be adding more configuration to work around these bugs which could cause additional issues. The fontconfig docs are tough: https://www.freedesktop.org/software/fontconfig/fontconfig-user.html

from emojione-color-font.

As a workaround for some issues, I have managed to follow this hint to install tex-gyre-fonts and resolve several issues. Almost all of the wrongly substituted fonts are now correctly substituted with TexGyre (or the appropriate otherwise font). However, the outstanding issue for me is that ZapfDingbats, Symbol are still wrongly substituted by Bitstream Vera and the only way to resolve this is to actually install the Adobe fonts. Not sure, if TexGyre will give you any extra clues.

from emojione-color-font.

No workaround is needed. Adobe's recommended solution is to embed the fonts. Your link says that also:

You may also experience similar problem when you open a PDF which requires Helvetica but does not have it embedded for viewing.

from emojione-color-font.

The workaround is needed when I want to read someone elses PDF that doesn't embed the fonts.

from emojione-color-font.

Ah ha! Haha, good point :D

from emojione-color-font.