Bug 228153 - REGRESSION(r280077): [GTK] It caused 155 new test failures
Summary: REGRESSION(r280077): [GTK] It caused 155 new test failures
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebKitGTK (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-07-21 11:58 PDT by Carlos Alberto Lopez Perez
Modified: 2021-09-15 14:02 PDT (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Carlos Alberto Lopez Perez 2021-07-21 11:58:21 PDT Comment hidden (obsolete)
Comment 1 Carlos Alberto Lopez Perez 2021-07-21 12:00:29 PDT
Commit r280077 caused 155 new failures on GTK.

The failures are related with slightly different rendering of the fonts that cause the ref-tests to fail (the ref-tests needs to match pixel-by-pixel the test)


This is the list of new failures:
  accessibility/math-multiscript-attributes.html [ Failure ]
  css1/font_properties/font.html [ Failure ]
  css1/pseudo/multiple_pseudo_elements.html [ Failure ]
  css2.1/t051201-c23-first-line-00-b.html [ Failure ]
  css2.1/t1508-c527-font-04-b.html [ Failure ]
  css2.1/t1508-c527-font-05-b.html [ Failure ]
  css2.1/t1508-c527-font-07-b.html [ Failure ]
  css2.1/t1508-c527-font-10-c.html [ Failure ]
  css3/font-feature-font-face-local.html [ ImageOnlyFailure ]
  css3/masking/mask-base64.html [ ImageOnlyFailure ]
  fast/block/lineboxcontain/block-glyphs-replaced.html [ Failure ]
  fast/block/lineboxcontain/glyphs.html [ Failure ]
  fast/css3-text/css3-text-decoration/text-decoration-skip/text-decoration-skip-ink.html [ ImageOnlyFailure ]
  fast/css/css2-system-fonts.html [ Failure ]
  fast/css-generated-content/quotes-lang-2.html [ ImageOnlyFailure ]
  fast/css-generated-content/quotes-lang-case-insensitive.html [ ImageOnlyFailure ]
  fast/css-generated-content/quotes-xml-lang.xhtml [ ImageOnlyFailure ]
  fast/repaint/japanese-rl-selection-clear.html [ Failure ]
  fast/repaint/japanese-rl-selection-repaint.html [ Failure ]
  fast/selectors/lang-dynamic.html [ ImageOnlyFailure ]
  fast/text/backslash-to-yen-sign-euc.html [ Failure ]
  fast/text/international/vertical-text-glyph-test.html [ Failure ]
  fast/text/international/vertical-text-metrics-test.html [ Failure ]
  fast/text/text-transform-turkish-and-azeri.html [ ImageOnlyFailure ]
  fast/writing-mode/border-vertical-lr.html [ Failure ]
  fast/writing-mode/japanese-lr-selection.html [ Failure ]
  fast/writing-mode/japanese-lr-text.html [ Failure ]
  fast/writing-mode/japanese-rl-selection.html [ Failure ]
  fast/writing-mode/japanese-rl-text.html [ Failure ]
  imported/blink/fast/css/case-sensitive-003.xhtml [ ImageOnlyFailure ]
  imported/blink/fast/css/case-sensitive-004.xhtml [ ImageOnlyFailure ]
  imported/blink/fast/text/text-transform-lower-greek.html [ ImageOnlyFailure ]
  imported/blink/svg/as-image/default-font-size.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-align/distribution-values/space-evenly-001.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-contain/contain-layout-001.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-contain/contain-layout-002.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-contain/contain-layout-003.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-contain/contain-layout-004.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-contain/contain-layout-005.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-contain/contain-layout-baseline-001.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-contain/contain-paint-002.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-contain/contain-size-001.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-contain/contain-size-002.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-contain/contain-size-003.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-grid/abspos/absolute-positioning-changing-containing-block-001.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-grid/abspos/grid-item-absolute-positioning-dynamic-001.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-grid/abspos/grid-positioned-item-dynamic-change-001.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-grid/abspos/grid-positioned-item-dynamic-change-002.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-grid/abspos/grid-positioned-item-dynamic-change-003.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-armenian.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-decimal.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-decimal-leading-zero.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-greek.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-latin.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-roman.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-upper-latin.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-upper-roman.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-pseudo/first-line-and-marker.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/hyphens/hyphens-none-011.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-004.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-005.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-006.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-007.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-008.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-009.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-010.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-011.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-012.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-013.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-014.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-015.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-016.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-017.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-and-white-space-001.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-and-white-space-002.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-and-white-space-003.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-and-white-space-005.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-and-white-space-008.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-and-white-space-009.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-overrides-uax-behavior-001.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-overrides-uax-behavior-002.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-overrides-uax-behavior-003.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-overrides-uax-behavior-004.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-overrides-uax-behavior-005.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-overrides-uax-behavior-006.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-overrides-uax-behavior-007.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-overrides-uax-behavior-008.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-overrides-uax-behavior-009.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-overrides-uax-behavior-010.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-overrides-uax-behavior-011.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-overrides-uax-behavior-012.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/line-break/line-break-anywhere-overrides-uax-behavior-015.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/overflow-wrap/overflow-wrap-break-word-009.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/overflow-wrap/overflow-wrap-break-word-010.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/word-break/word-break-break-all-029.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/word-break/word-break-break-all-030.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-text/word-break/word-break-break-all-inline-010.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-ui/outline-003.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-ui/outline-004.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-ui/outline-008.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-ui/outline-009.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-ui/outline-010.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-ui/outline-011.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-ui/outline-012.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-ui/outline-019.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-values/calc-rem-lang.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-writing-modes/writing-mode-initial.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/selectors/focus-within-shadow-006.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/selectors/scope-without-scoping.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/global-attributes/lang-xmllang-01.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-001a.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-001b.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-001c.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-002a.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-002b.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-002c.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-003a.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-003b.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-003c.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-004a.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-004b.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-004c.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-005a.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-005b.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-005c.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-006a.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-006b.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-006c.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-007a.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-007b.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-007c.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-008a.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-008b.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-008c.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-009a.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-009b.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-009c.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/editing/editing-0/spelling-and-grammar-checking/spelling-markers-007.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/html/editing/editing-0/spelling-and-grammar-checking/spelling-markers-009.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/mathml/relations/css-styling/ignored-properties-001.html [ Failure ]
  imported/w3c/web-platform-tests/mathml/relations/css-styling/padding-border-margin/border-002.html [ Failure ]
  imported/w3c/web-platform-tests/mathml/relations/css-styling/padding-border-margin/padding-002.html [ Failure ]
  imported/w3c/web-platform-tests/svg/import/text-intro-03-b-manual.svg [ Failure ]
  mathml/opentype/horizontal.html [ Failure ]
  mathml/opentype/horizontal-munderover.html [ Failure ]
  mathml/opentype/large-operators.html [ Failure ]
  mathml/opentype/opentype-stretchy-horizontal.html [ Failure ]
  mathml/opentype/vertical.html [ Failure ]
  mathml/presentation/bug159513.html [ Failure ]
  mathml/presentation/mo-stretch.html [ Failure ]
  mathml/presentation/multiscripts-equivalence.html [ ImageOnlyFailure ]
  mathml/presentation/roots.xhtml [ Failure ]
  platform/gtk/fonts/webkit-font-smoothing.html [ ImageOnlyFailure ]
  svg/batik/text/verticalText.svg [ Failure ]
  svg/text/offset-square-svg-font.html [ ImageOnlyFailure ]


See here the results

- https://people.igalia.com/clopez/wkbug/227854/layout-test-results/results.html
- downloadable at: https://people.igalia.com/clopez/wkbug/227854/layout-test-results.tar.xz

See for example this two cases where the different rendering of the font (on the test and the ref-test) is appreciated

- https://people.igalia.com/clopez/wkbug/227854/layout-test-results/platform/gtk/fonts/webkit-font-smoothing-diffs.html
- https://people.igalia.com/clopez/wkbug/227854/layout-test-results/imported/w3c/web-platform-tests/css/css-contain/contain-layout-004-diffs.html

There also tests with text failures like:

- https://people.igalia.com/clopez/wkbug/227854/layout-test-results/css2.1/t1508-c527-font-10-c-pretty-diff.html
Comment 2 Alexander Mikhaylenko 2021-07-21 12:34:19 PDT
So I see Tools/TestWebKitAPI/Tests/WebKitGLib/TestWebKitSettings.cpp and Tools/WebKitTestRunner/InjectedBundle/gtk/ActivateFontsGtk.cpp change the settings.

The former seems to run in the UI process and so it didn't work before and the change fixed it and the tests that relied on it need updates.

The latter though is broken now - the web process will pull settings from the UI process and those settings should be set from the UI process instead.
Comment 3 Carlos Alberto Lopez Perez 2021-07-21 13:34:21 PDT
Updated test expectations on r280159
Comment 4 Carlos Alberto Lopez Perez 2021-09-15 12:37:19 PDT
On r282195 a new test was added imported/w3c/web-platform-tests/css/css-pseudo/first-letter-with-quote.html which is passing on all platforms but GTK (it passes on WPE).

Looking at the failure diff I think it maybe very related to this bug.

See: https://people.igalia.com/clopez/wkbug/227854/extra-results-1/layout-test-results/results.html

It is the same type of diff.. a slightly different rendering of the font on the ref-test than on the test.


Likely also bug 227231 is also related to this.
Comment 5 Carlos Alberto Lopez Perez 2021-09-15 14:02:28 PDT
Test expectations updated in r282471