Enabling the SVG -> OTF Font Converter in the GTK+ port r1978134 caused 96 new failures. This is the complete list of this 96 tests http://sprunge.us/MHRh that started to fail in r1978134.
I have been checking the differences and I rebaselined in r198993 those tests that looked to produce an identical (or almost identical) output image. However, there are 16 tests that look like real failures: * 4 of them were already reported in bug 137096 * 12 of them are new failures: Regressions: Unexpected text-only failures (12) fast/text/svg-font-face-with-kerning.html [ Failure ] svg/W3C-SVG-1.1-SE/types-dom-06-f.svg [ Failure ] svg/W3C-SVG-1.1/text-text-05-t.svg [ Failure ] svg/W3C-SVG-1.1/text-text-06-t.svg [ Failure ] svg/custom/glyph-selection-lang-attribute.svg [ Failure ] svg/text/kerning.svg [ Failure ] svg/text/multichar-glyph.svg [ Failure ] svg/text/text-hkern.svg [ Failure ] svg/text/text-overflow-ellipsis-svgfont-kerning-ligatures.html [ Failure ] svg/text/text-text-04-t.svg [ Failure ] svg/text/text-text-06-t.svg [ Failure ] svg/text/text-vkern.svg [ Failure ] Specifically the kerning with SVG fonts seems broken. I have updated the TestExpectations accordingly on r198993.
(In reply to comment #0) > Enabling the SVG -> OTF Font Converter in the GTK+ port r1978134 caused 96 > new failures. > > This is the complete list of this 96 tests http://sprunge.us/MHRh that > started to fail in r1978134. Sorry, I was meaning since r197814 <http://trac.webkit.org/r197814> (not r1978134)
(In reply to comment #1) > I have been checking the differences and I rebaselined in r198993 those > tests that looked to produce an identical (or almost identical) output image. > > However, there are 16 tests that look like real failures: > > * 4 of them were already reported in bug 137096 > * 12 of them are new failures: > > Regressions: Unexpected text-only failures (12) > fast/text/svg-font-face-with-kerning.html [ Failure ] > svg/W3C-SVG-1.1-SE/types-dom-06-f.svg [ Failure ] > svg/W3C-SVG-1.1/text-text-05-t.svg [ Failure ] > svg/W3C-SVG-1.1/text-text-06-t.svg [ Failure ] > svg/custom/glyph-selection-lang-attribute.svg [ Failure ] > svg/text/kerning.svg [ Failure ] > svg/text/multichar-glyph.svg [ Failure ] > svg/text/text-hkern.svg [ Failure ] > svg/text/text-overflow-ellipsis-svgfont-kerning-ligatures.html [ Failure ] > svg/text/text-text-04-t.svg [ Failure ] > svg/text/text-text-06-t.svg [ Failure ] > svg/text/text-vkern.svg [ Failure ] > > This are the results for those tests with pixel tests enabled: http://people.igalia.com/clopez/wkbug/156148/layout-test-results/results.html The two that has missing results: * svg/text/text-overflow-ellipsis-svgfont-kerning-ligatures.html fails clearly (no ellipsis is shown). Compare it with the output from the mac port <http://trac.webkit.org/browser/trunk/LayoutTests/platform/mac/svg/text/text-overflow-ellipsis-svgfont-kerning-ligatures-expected.png?rev=152653> * svg/text/text-hkern.svg gives a very different image compared with the one for the mac port <http://trac.webkit.org/browser/trunk/LayoutTests/platform/mac/svg/text/text-hkern-expected.png?rev=199005>
(In reply to comment #1) > I have been checking the differences and I rebaselined in r198993 those > tests that looked to produce an identical (or almost identical) output image. > > However, there are 16 tests that look like real failures: > > * 4 of them were already reported in bug 137096 > * 12 of them are new failures: > > Regressions: Unexpected text-only failures (12) > fast/text/svg-font-face-with-kerning.html [ Failure ] > svg/W3C-SVG-1.1-SE/types-dom-06-f.svg [ Failure ] > svg/W3C-SVG-1.1/text-text-05-t.svg [ Failure ] > svg/W3C-SVG-1.1/text-text-06-t.svg [ Failure ] > svg/custom/glyph-selection-lang-attribute.svg [ Failure ] > svg/text/kerning.svg [ Failure ] > svg/text/multichar-glyph.svg [ Failure ] > svg/text/text-hkern.svg [ Failure ] > svg/text/text-overflow-ellipsis-svgfont-kerning-ligatures.html [ Failure ] > svg/text/text-text-04-t.svg [ Failure ] > svg/text/text-text-06-t.svg [ Failure ] > svg/text/text-vkern.svg [ Failure ] > > > Specifically the kerning with SVG fonts seems broken. > > I have updated the TestExpectations accordingly on r198993. This is probably caused by https://bugs.webkit.org/show_bug.cgi?id=137204. Does GTK's text shaping engine support the 'kern' table if it exists?
(In reply to comment #4) > > Does GTK's text shaping engine support the 'kern' table if it exists? The documentation says this: """ Note that OpenType fonts (OTF) provide two distinct mechanisms for kerning, using the ‘kern’ and ‘GPOS’ tables, respectively, which are part of the OTF files. Older fonts only contain the former, while recent fonts contain both tables or even ‘GPOS’ data only. FreeType only supports kerning via the (rather simple) ‘kern’ table. For the interpretation of kerning data in the (highly sophisticated) ‘GPOS’ table you need a higher-level library like ICU or HarfBuzz since it can be context dependent (this is, the kerning may vary depending on the position within a text string, for example). """ http://www.freetype.org/freetype2/docs/glyphs/glyphs-4.html And I see that there are two standards or versions for the 'kern' table: """ Apple has extended the definition of the 'kern' table to provide additional functionality. The Apple extensions are not supported on Windows. Fonts intended for cross-platform use or for the Windows platform in general should conform to the 'kern' table format specified here. """ https://www.microsoft.com/typography/otspec/kern.htm I guess that you are using the Apple dialect for the 'kern' table in the OTF font generated??? How can I save the OTF font that is generated from the SVG one <LayoutTests/svg/custom/resources/SVGFreeSans.svg> to a file??? That way I can try to check what freetype's ftvalid/ftdump commands says about it.
I don't understand the interaction between how we use FreeType and Harfbuzz, but keep in mind that we do use and depend on both.
(In reply to comment #6) > I don't understand the interaction between how we use FreeType and Harfbuzz, > but keep in mind that we do use and depend on both. (In reply to comment #6) > I don't understand the interaction between how we use FreeType and Harfbuzz, > but keep in mind that we do use and depend on both. We follow the specification in the Microsoft docs (meaning: we don't use any Apple-specific extensions). The best way to save the generated file is to modify convertSVGToOTFFont() in SVGToOTFFontConversion.cpp. It returns an Optional<Vector<char>>. Just before returning, that function could write out the vector to a file. (Keep in mind you'll have to modify the WebKit2 sandbox rules to allow the Web Process access to the disk)
(In reply to comment #6) > I don't understand the interaction between how we use FreeType and Harfbuzz, > but keep in mind that we do use and depend on both. My guess is that Harfbuzz is used for complex text shaping only. FreeType is (probably) only used for rendering the glyphs once they have been shaped. (But I don't know for sure, that's just a guess)
(In reply to comment #7) > (In reply to comment #6) > > I don't understand the interaction between how we use FreeType and Harfbuzz, > > but keep in mind that we do use and depend on both. > (In reply to comment #6) > > I don't understand the interaction between how we use FreeType and Harfbuzz, > > but keep in mind that we do use and depend on both. > > We follow the specification in the Microsoft docs (meaning: we don't use any > Apple-specific extensions). > > The best way to save the generated file is to modify convertSVGToOTFFont() > in SVGToOTFFontConversion.cpp. It returns an Optional<Vector<char>>. Just > before returning, that function could write out the vector to a file. (Keep > in mind you'll have to modify the WebKit2 sandbox rules to allow the Web > Process access to the disk) Not sure if the WebKit2 sandbox is not enabled/working on the GTK+ port or if it is enabled but allowing write to /tmp, but I was able to dump the OTF file generated to disk with just something like this http://people.igalia.com/clopez/wkbug/156148/SVGToOTFFontConversion_todisk.diff I uploaded that generated OTF font to here http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/SVGFreeSans.otf I also converted the original SVG font to OTF using onlinefontconverter.com and uploaded it to here http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/SVGFreeSans_onlinefontconverter.com.otf And I have compared the two OTF fonts with some utilities and this are the main differences I can see: 1) Difference in tables - The OTF font generated by WebKit has a kern table. But the one generated by onlinefontconverter.com has instead a GPOS one. - List of tables for OTF font generated by WebKit: http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/SVGFreeSans_ttxtables.txt - List of tables for OTF font generated by onlinefontconverter: http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/SVGFreeSans_onlinefontconverter.com_ttxtables.txt 2) Converting the OTF font to ttx format (a short of xml that the ttx utility from fonttools produces) causes errors with the WebKit otf font but not with the onlinefontconverter OTF one: - The conversion from the OTF font generated by WebKit gives this errors: http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/SVGFreeSans_ttxdetails_stderr.txt and generated this XML http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/SVGFreeSans_ttxdetails.xml - The conversion from the OTF font generated by onlinefontconverter gives no error: http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/SVGFreeSans_onlinefontconverter.com_ttxdetails_stderr.txt and generates this XML http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/SVGFreeSans_onlinefontconverter.com_ttxdetails.xml - Trying to convert the xml back to otf is only possible with the OTF font generated by onlinefontconverter, with the WebKit one the ttx utility aborts with an ugly backtrace. 3) To conclude my tests, I have also created this test page: http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/onlinefontconverter.com/test_svg_otf.html - And generated 3 screenshots at http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/screenshots/ As you can see there is a clear difference between the results rendered with the OTF font generated by WebKit and the one generated by onlinefontconverter. This difference is also appreciable in safari from current webkit trunk. So.... I may be wrong, but I think there is something wrong with the OTF font that WebKit generates.
(In reply to comment #9) > 3) To conclude my tests, I have also created this test page: > > http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/onlinefontconverter. > com/test_svg_otf.html > ^^^ right url: http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/test_svg_otf.html > - And generated 3 screenshots at > http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/screenshots/ > > As you can see there is a clear difference between the results rendered with > the OTF font generated by WebKit and the one generated by > onlinefontconverter. > > This difference is also appreciable in safari from current webkit trunk. > > So.... I may be wrong, but I think there is something wrong with the OTF > font that WebKit generates.
(In reply to comment #9) > Not sure if the WebKit2 sandbox is not enabled/working on the GTK+ port Our sandbox is disabled by default because it is extremely fragile. I am most likely going to delete it.
(In reply to comment #9) > (In reply to comment #7) > > (In reply to comment #6) > > > I don't understand the interaction between how we use FreeType and Harfbuzz, > > > but keep in mind that we do use and depend on both. > > (In reply to comment #6) > > > I don't understand the interaction between how we use FreeType and Harfbuzz, > > > but keep in mind that we do use and depend on both. > > > > We follow the specification in the Microsoft docs (meaning: we don't use any > > Apple-specific extensions). > > > > The best way to save the generated file is to modify convertSVGToOTFFont() > > in SVGToOTFFontConversion.cpp. It returns an Optional<Vector<char>>. Just > > before returning, that function could write out the vector to a file. (Keep > > in mind you'll have to modify the WebKit2 sandbox rules to allow the Web > > Process access to the disk) > > Not sure if the WebKit2 sandbox is not enabled/working on the GTK+ port > or if it is enabled but allowing write to /tmp, but I was able to dump the > OTF > file generated to disk with just something like this > http://people.igalia.com/clopez/wkbug/156148/SVGToOTFFontConversion_todisk. > diff > > I uploaded that generated OTF font to here > http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/SVGFreeSans.otf > > I also converted the original SVG font to OTF using onlinefontconverter.com > and uploaded it to here > http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/ > SVGFreeSans_onlinefontconverter.com.otf > > > > And I have compared the two OTF fonts with some utilities and this are the > main differences I can see: > > 1) Difference in tables > > - The OTF font generated by WebKit has a kern table. But the one generated > by onlinefontconverter.com has instead a GPOS one. > > - List of tables for OTF font generated by WebKit: > http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/ > SVGFreeSans_ttxtables.txt > - List of tables for OTF font generated by onlinefontconverter: > http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/ > SVGFreeSans_onlinefontconverter.com_ttxtables.txt > > 2) Converting the OTF font to ttx format (a short of xml that the ttx > utility from fonttools produces) causes errors with the WebKit otf font but > not with the onlinefontconverter OTF one: > > - The conversion from the OTF font generated by WebKit gives this errors: > http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/ > SVGFreeSans_ttxdetails_stderr.txt and generated this XML > http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/ > SVGFreeSans_ttxdetails.xml > - The conversion from the OTF font generated by onlinefontconverter gives no > error: > http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/ > SVGFreeSans_onlinefontconverter.com_ttxdetails_stderr.txt and generates this > XML > http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/ > SVGFreeSans_onlinefontconverter.com_ttxdetails.xml > > - Trying to convert the xml back to otf is only possible with the OTF font > generated by onlinefontconverter, with the WebKit one the ttx utility aborts > with an ugly backtrace. > > > 3) To conclude my tests, I have also created this test page: > > http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/onlinefontconverter. > com/test_svg_otf.html > > - And generated 3 screenshots at > http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/screenshots/ > > As you can see there is a clear difference between the results rendered with > the OTF font generated by WebKit and the one generated by > onlinefontconverter. > > This difference is also appreciable in safari from current webkit trunk. > > So.... I may be wrong, but I think there is something wrong with the OTF > font that WebKit generates. Wow, really great work! Given that the two converters are entirely different codebases, it's expected that the list of tables will be different. When I look at the table lists, I see: - onlinefontconverter includes a "FontForge timestamp" table, which is unused by all text engines (that I'm aware of) - onlinefontconverter includes a 'GDEF' table. It doesn't appear that there is any information in that table which is also supplied in SVG, so I don't think I have any choice but to leave it out. - onlinefontconverter includes a 'GPOS' table, which is the table used by Windows (and maybe Harfbuzz/FreeType??) to perform kerning (instead of 'kern'). - WebKit includes 'VORG' 'vhea' and 'vmtx' tables which are used for laying out vertical text (presumably onlinefontconverter doesn't care about vertical text?) (SVG fonts allow for vertical text). Looking at the ttx output, I see: - It doesn't like our timestamps. WebKit hardcodes these timestamps to 0 because they are unused by all text engines (that I'm aware of). - It thinks our 'OS/2' table is too long. There are different versions of the 'OS/2' table. The TrueType spec only lists version 0, and mentions that Apple doesn't use this table. The OpenType spec lists versions 0, 1, 2, and 3. WebKit implements version 2, because that is what the Ahem.ttf in our LayoutTests/resources directory uses. The ttx tool probably doesn't understand version 2 of the table. Looking at your demo content on OS X, I see two things: - I think you made a typo in the section for "With WebKit converted svg->otf font:". My browser gives a 404 with the following URL: http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/SVGFreeSans.otg - The kerning appears to work on the other 3 sections (it seems most observable between the "T" and the "e" in "Team") However, your screenshots from your GTK system do show r197813 causing kerning to stop working. So, after all that investigation, I have these thoughts: - It is definitely very reasonable for there to be a problem in the font converter. We are writing out fonts byte-per-byte, and the format is ugly enough / implementations are finicky enough that this is definitely possible (perhaps even probable?) - Because of the second sentence of the above item, the ttx tool isn't necessarily the best signal of problems. Every parser of these fonts will work differently, and I'm primarily interested in the browser's layout / rendering of text. - Therefore, instead of saying "there is something wrong" I think we need to focus on specific problems, and investigate them. It sounds like, in this situation, the specific problem is "kerning stopped working." - All the investigation above is consistent with the known bug that the converter outputs 'kern' tables instead of 'GPOS' tables. This bug also caused kerning to stop working on Windows. - Making the converter write out GPOS tables instead of kern tables is not too difficult, I just haven't done it yet. I intend to do it in the near(-ish) future. So, after all that, the question we need to ask ourselves is: Do we need to revert the patches until this bug is fixed? I believe the answer to that question is "no" for the following reason: You can consider the expressiveness of OpenType and SVG fonts like a Venn diagram. There are OpenType fonts which cannot be expressed in SVG, and there are SVG fonts which cannot be expressed in OpenType. Some examples of things which an SVG font can contain but an OpenType font can't are animations and (non-bitmapped) gradients. The reason why it is valuable to convert these fonts is that the payoff is worth the loss of expressiveness. The payoff is huge (huge performance increases, allowing for subpixel antialiasing, removing a whole class of security bugs) and the usage and impact of this lost expressiveness is low. Granted, kerning in SVG fonts is used a non-zero amount on the web, but the combination of 1. its low usage, 2. its low(-ish) visual impact, and 3. the fact that I intend to fix this bug soon make the case very clear.
(In reply to comment #12) > > Given that the two converters are entirely different codebases, it's > expected that the list of tables will be different. > > When I look at the table lists, I see: > - onlinefontconverter includes a "FontForge timestamp" table, which is > unused by all text engines (that I'm aware of) > - onlinefontconverter includes a 'GDEF' table. It doesn't appear that there > is any information in that table which is also supplied in SVG, so I don't > think I have any choice but to leave it out. > - onlinefontconverter includes a 'GPOS' table, which is the table used by > Windows (and maybe Harfbuzz/FreeType??) to perform kerning (instead of > 'kern'). > - WebKit includes 'VORG' 'vhea' and 'vmtx' tables which are used for laying > out vertical text (presumably onlinefontconverter doesn't care about > vertical text?) (SVG fonts allow for vertical text). > > > Looking at the ttx output, I see: > - It doesn't like our timestamps. WebKit hardcodes these timestamps to 0 > because they are unused by all text engines (that I'm aware of). > - It thinks our 'OS/2' table is too long. There are different versions of > the 'OS/2' table. The TrueType spec only lists version 0, and mentions that > Apple doesn't use this table. The OpenType spec lists versions 0, 1, 2, and > 3. WebKit implements version 2, because that is what the Ahem.ttf in our > LayoutTests/resources directory uses. The ttx tool probably doesn't > understand version 2 of the table. > > > Looking at your demo content on OS X, I see two things: > - I think you made a typo in the section for "With WebKit converted svg->otf > font:". My browser gives a 404 with the following URL: > http://people.igalia.com/clopez/wkbug/156148/SVGFreeSans/SVGFreeSans.otg > - The kerning appears to work on the other 3 sections (it seems most > observable between the "T" and the "e" in "Team") > Ups. So the test and the screenshots were all buggy because it was trying to load a 404 (SVGFreeSans.otg) rather than SVGFreeSans.otf. I have fixed the html and created new screenshots. > > However, your screenshots from your GTK system do show r197813 causing > kerning to stop working. > Yes, it seems. > > So, after all that investigation, I have these thoughts: > - It is definitely very reasonable for there to be a problem in the font > converter. We are writing out fonts byte-per-byte, and the format is ugly > enough / implementations are finicky enough that this is definitely possible > (perhaps even probable?) > - Because of the second sentence of the above item, the ttx tool isn't > necessarily the best signal of problems. Every parser of these fonts will > work differently, and I'm primarily interested in the browser's layout / > rendering of text. > - Therefore, instead of saying "there is something wrong" I think we need to > focus on specific problems, and investigate them. It sounds like, in this > situation, the specific problem is "kerning stopped working." The previous broken test tricked me in thinking the generated OTF font was broken. Now after fixing the test, the screenshot with safari looks perfect. So it seems at least Safari can read correctly the generated font. > - All the investigation above is consistent with the known bug that the > converter outputs 'kern' tables instead of 'GPOS' tables. This bug also > caused kerning to stop working on Windows. > - Making the converter write out GPOS tables instead of kern tables is not > too difficult, I just haven't done it yet. I intend to do it in the > near(-ish) future. > > So, after all that, the question we need to ask ourselves is: Do we need to > revert the patches until this bug is fixed? > I don't think so. I don't think is a major problem to have kerning broken with SVG fonts. After all SVG fonts are not very used on web pages. > I believe the answer to that question is "no" for the following reason: > > You can consider the expressiveness of OpenType and SVG fonts like a Venn > diagram. There are OpenType fonts which cannot be expressed in SVG, and > there are SVG fonts which cannot be expressed in OpenType. Some examples of > things which an SVG font can contain but an OpenType font can't are > animations and (non-bitmapped) gradients. The reason why it is valuable to > convert these fonts is that the payoff is worth the loss of expressiveness. > The payoff is huge (huge performance increases, allowing for subpixel > antialiasing, removing a whole class of security bugs) and the usage and > impact of this lost expressiveness is low. Granted, kerning in SVG fonts is > used a non-zero amount on the web, but the combination of 1. its low usage, > 2. its low(-ish) visual impact, and 3. the fact that I intend to fix this > bug soon make the case very clear. Thanks for the comments.