Bug 81544 – Layout Test fast/text/international/font-fallback-to-common-script.html has flaky image results on all Chromium bots

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Levi Weintraub]

fast/text/international/font-fallback-to-common-script.html is failing often on image results in all three major platforms on Chromium. It's possible that only the text results matter...

[Matt Falkenhagen]

I wonder if it's some race condition involving the lines:

  window.internals.settings.setStandardFontFamily("Times", "Hans");
  window.internals.settings.setStandardFontFamily("", "Hans");

to test that removal of "Times" with "" works. Is there a way to ensure the test runs after the effects of the lines are done?

The other reftests for font settings don't seem to be flaky (fast/text/international/content-language-font-selection.html, fast/text/international/content-language-with-subtags.html, fast/text/international/locale-sensitive-fonts.html).

I can try removing the "Times" line and see if it helps.

[Matt Falkenhagen]

Removing the "Times" setting doesn't seem to help. It still failed when I ran on try bots.

It's a mystery why this test is flaky and similar tests aren't:

http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=fast%2Ftext%2Finternational%2Flang-sensitive-fonts.html&showExpectations=true
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=fast%2Ftext%2Finternational%2Fcontent-language-font-selection.html&showExpectations=true

It is odd that the image results are blank, with the "PASS, if no..." line showing. Maybe somehow the image is grabbed before it had a chance to finish rendering.

[Matt Falkenhagen]

Created attachment 135503 [details]
Patch

[Matt Falkenhagen]

I don't know why, but renaming the test seems to allow it to pass consistently on the trybots. Uploaded a patch to do that.

[Kent Tamura]

(In reply to comment #4)
> I don't know why, but renaming the test seems to allow it to pass consistently on the trybots. Uploaded a patch to do that.

So, I guess a test running before font-fallback-to-common-script.html leaves unexpected setting, and a test running after font-fallback-to-common-script.html before lang-sensitive-fonts-fallback.html fixes the setting.

[Matt Falkenhagen]

It seems the "PASS, if no exception or crash in debug" in the image result comes from the previous test embed-bidi-style-in-isolate-crash.html. I've filed bug 83260.

[Matt Falkenhagen]

Created attachment 135791 [details]
Patch

[Kent Tamura]

Comment on attachment 135791 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=135791&action=review

> LayoutTests/platform/chromium/test_expectations.txt:-3296
> -BUGWK73549 : fast/text/international/hebrew-vowels.html = IMAGE+TEXT IMAGE TEXT PASS

We need to do rebaseline for this before landing this patch.
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#showExpectations=true&showLargeExpectations=true&tests=hebrew-vowels.html

[Kent Tamura]

Comment on attachment 135791 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=135791&action=review

>> LayoutTests/platform/chromium/test_expectations.txt:-3296
>> -BUGWK73549 : fast/text/international/hebrew-vowels.html = IMAGE+TEXT IMAGE TEXT PASS
> 
> We need to do rebaseline for this before landing this patch.
> http://test-results.appspot.com/dashboards/flakiness_dashboard.html#showExpectations=true&showLargeExpectations=true&tests=hebrew-vowels.html

I fixed this by http://trac.webkit.org/changeset/113306.
We can go ahead.

[WebKit Review Bot]

Comment on attachment 135791 [details]
Patch

Clearing flags on attachment: 135791

Committed r113308: <http://trac.webkit.org/changeset/113308>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.