Bug 47727 – [GTK] mathml flaky tests after adding SystemFonts to the theme

• Status: RESOLVED
• Resolution: FIXED
• Component: WebKitGTK

[Alejandro G. Castro]

After the commit fixing the fonts in the forms these tests started to be flaky, mostly in the debug bots.

mathml/presentation/fenced.xhtml
mathml/presentation/roots.xhtml
mathml/presentation/subsup.xhtml

http://trac.webkit.org/changeset/69818

[Alex Milowski]

I believe bug 47774 has taken care of this.  Can you check again?

[Martin Robinson]

As I understand it, the tests differed between the bots, so we may need some extra machinery in the DRT setup to ensure that STIX fonts load.

[Alex Milowski]

We don't use the STIX fonts yet.  That has yet to be configured.  We'd need to setup a check/download step in the run-webkit-tests script.

[Martin Robinson]

(In reply to comment #3)
> We don't use the STIX fonts yet.  That has yet to be configured.  We'd need to setup a check/download step in the run-webkit-tests script.

The MathML stylesheet specifies them if I understand correctly. I believe that some of the bots have them and some of them don't or they have different versions.

[Alex Milowski]

OK.  I understand the issue now.  We want them all to have it.  I'd like the test infrastructure to guarantee this.

[Alejandro G. Castro]

(In reply to comment #1)
> I believe bug 47774 has taken care of this.  Can you check again?

I've ran the tests locally and they still have issues with current results. I've checked also the STIX fonts in the bots a apparently the debug bots (32bits and 64bits) have it, do I have to look for some version or other specific information?


bb-webkit-32 ~ # fc-list | grep STIX
STIXIntegralsUp:style=Regular
STIXIntegralsSm:style=Regular
STIXSizeFourSym:style=Bold
STIXNonUnicode:style=Bold
STIXVariants:style=Regular
STIXIntegralsSm:style=Bold
STIXGeneral:style=Bold
STIXIntegralsUp:style=Bold
STIXGeneral:style=Italic
STIXVariants:style=Bold
STIXIntegralsUpD:style=Bold
STIXNonUnicode:style=Italic
STIXGeneral:style=Regular
STIXSizeTwoSym:style=Bold
STIXGeneral:style=Bold Italic
STIXNonUnicode:style=Regular
STIXNonUnicode:style=Bold Italic
STIXSizeOneSym:style=Regular
STIXSizeOneSym:style=Bold
STIXSizeFourSym:style=Regular
STIXIntegralsUpSm:style=Bold
STIXSizeTwoSym:style=Regular
STIXIntegralsD:style=Bold
STIXIntegralsUpD:style=Regular
STIXIntegralsD:style=Regular
STIXSizeThreeSym:style=Regular
STIXSizeFiveSym:style=Regular
STIXIntegralsUpSm:style=Regular
STIXSizeThreeSym:style=Bold


bb-webkit-64 ~ # fc-list | grep STIX
STIXIntegralsUp:style=Regular
STIXIntegralsSm:style=Regular
STIXSizeFourSym:style=Bold
STIXNonUnicode:style=Bold
STIXVariants:style=Regular
STIXIntegralsSm:style=Bold
STIXGeneral:style=Bold
STIXIntegralsUp:style=Bold
STIXGeneral:style=Italic
STIXVariants:style=Bold
STIXIntegralsUpD:style=Bold
STIXNonUnicode:style=Italic
STIXGeneral:style=Regular
STIXSizeTwoSym:style=Bold
STIXGeneral:style=Bold Italic
STIXNonUnicode:style=Regular
STIXNonUnicode:style=Bold Italic
STIXSizeOneSym:style=Regular
STIXSizeOneSym:style=Bold
STIXSizeFourSym:style=Regular
STIXIntegralsUpSm:style=Bold
STIXSizeTwoSym:style=Regular
STIXIntegralsD:style=Bold
STIXIntegralsUpD:style=Regular
STIXIntegralsD:style=Regular
STIXSizeThreeSym:style=Regular
STIXSizeFiveSym:style=Regular
STIXIntegralsUpSm:style=Regular
STIXSizeThreeSym:style=Bold

[Alejandro G. Castro]

At this point there is a new mathml/presentation test that it is failing in the 32bots (debug and release) but it is not failing in the 64bits debug bot, at least once. I'm going to add it to the Skipped file until we find a solution:

mathml/presentation/fenced-mi.xhtml

[Alex Milowski]

The tests are failing because the current results do not use the STIX fonts.  We need to make sure all the build bots have the STIX fonts.  This should get all sorted out shortly and we'll re-baseline the MathML tests with the STIX fonts.

[Martin Robinson]

Created attachment 71105 [details]
Ensure STIX fonts in DRT setup

[Alex Milowski]

(In reply to comment #9)
> Created an attachment (id=71105) [details]
> Ensure STIX fonts in DRT setup

I've looked at these results and they look pretty good.  The issue with the glyph stacking for the absolute sign (vertical bar) and integral sign are some STIX/GTK issue we'll need to work on.  I've filed a bug for the absolute sign already.  These have to do with the interaction between the glyph stacking algorithm and the font metrics.

[Alejandro G. Castro]

(In reply to comment #9)
> Created an attachment (id=71105) [details]
> Ensure STIX fonts in DRT setup

I've tested the patch locally running the mathml tests, the visual result look good but the expected results do not match. Apparently small pixel differences.

[Gustavo Noronha (kov)]

Comment on attachment 71105 [details]
Ensure STIX fonts in DRT setup

Let's do it.

[Martin Robinson]

Committed r70253: <http://trac.webkit.org/changeset/70253>

[Alejandro G. Castro]

I'm afraid this did not fix the issue mathml/presentation tests are still flaky.

[Alejandro G. Castro]

I'm skipping all the tests until we find the problem:

http://trac.webkit.org/changeset/70383

[Alejandro G. Castro]

Probably the fix in http://trac.webkit.org/changeset/72413 could solve this issue, we could try to unskip these tests to check it.

[Alex Milowski]

(In reply to comment #16)
> Probably the fix in http://trac.webkit.org/changeset/72413 could solve this issue, we could try to unskip these tests to check it.

That sounds like a good idea.  Is there any way to test this other than turning the test back on and waiting till things fail in the build bots?

[Alejandro G. Castro]

(In reply to comment #17)
> (In reply to comment #16)
> > Probably the fix in http://trac.webkit.org/changeset/72413 could solve this issue, we could try to unskip these tests to check it.
> 
> That sounds like a good idea.  Is there any way to test this other than turning the test back on and waiting till things fail in the build bots?

I'm afraid we can not 100% sure, I'm going to run the tests in a loop in my local machine and in case I can not reproduce it I'll unskip it.

[Alejandro G. Castro]

Unskipped, let's check if that works. http://trac.webkit.org/changeset/72670

[Alejandro G. Castro]

It did not fix the issue, I'm going to skip again and we have to check the issue. 

http://build.webkit.org/results/GTK Linux 64-bit Debug/r72674 (16327)/mathml/presentation/subsup-pretty-diff.html

[Eric Seidel (no email)]

Seems this should be closed?

[Alejandro G. Castro]

(In reply to comment #21)
> Seems this should be closed?

They are still in the Skipped file and flaky, so we still have to check the issue.

[Eric Seidel (no email)]

Attachment 71105 [details] was posted by a committer and has review+, assigning to Martin Robinson for commit.

[Martin Robinson]

Comitted as http://trac.webkit.org/changeset/70253

[Zan Dobersek]

The tests required updated baselines, which was done in r122704[1]. The baselines are almost identical to those for Mac port, and with the tests now passing with no flakiness visible I've removed their expectations in r123096[2]. Closing the bug.


[1] - http://trac.webkit.org/changeset/122704
[2] - http://trac.webkit.org/changeset/123096