Bug 289082 – [GTK] Images produced from a reftest look different from the corresponding documents opened in tabs

• Component: New Bugs
• Status: NEW

[Deleted User]

Created attachment 474439 [details]
Screenshot of the reference image

STR:

1. Tools/Scripts/import-w3c-tests -t web-platform-tests/html/canvas/element/manual/text/canvas.2d.lang.html
2. Open the following imported test in and editor and append some character, e.g. "Z" to the body: LayoutTests/imported/w3c/web-platform-tests/html/canvas/element/manual/text/canvas.2d.lang.html. That's done so that the test fails because of a mismatch so that the rendered images an
3. Run the test.
4. In the result browser window, open the reference file linked from the failing test in a new tab.
5. Click the "images" link of the failed test to show the expected image.

Expected: visually the results of step 4. and 5. should look the same.
Actual: they differ.

[Deleted User]

Created attachment 474440 [details]
Screenshot of the reference file opened in separate tab

[Deleted User]

Happened with GTK on Ubuntu 24.04 but may affect other platforms as well.

[Alexey Proskuryakov]

Generally, I wouldn't expect the output of test runner to match what's in the browser window. There are many different settings in the test runner, and fonts should be hand picked too. Additionally, WPT tests aren't designed to be opened as file:///, they need to be served with wptserve.

There is also a canonical server with these files at https://wpt.live/html/canvas/element/manual/text/canvas.2d.lang.html / https://wpt.live/html/canvas/element/manual/text/canvas.2d.lang-ref.html

Is there something in particular here that you find incorrect?

[Deleted User]

(In reply to Alexey Proskuryakov from comment #3)
> Generally, I wouldn't expect the output of test runner to match what's in
> the browser window. There are many different settings in the test runner,
> and fonts should be hand picked too. Additionally, WPT tests aren't designed
> to be opened as file:///, they need to be served with wptserve.
> 

Thanks for pointing that out.

> There is also a canonical server with these files at
> https://wpt.live/html/canvas/element/manual/text/canvas.2d.lang.html /
> https://wpt.live/html/canvas/element/manual/text/canvas.2d.lang-ref.html

Opening those links in MiniBrowser yields the same results as opening the documents via `file:///`.

> 
> Is there something in particular here that you find incorrect?

Yes. It seems the fonts which are used are different. This matters for tests where a font supports different renderings for different languages. The above mentioned reftest relies on the font supporting different renderings. If it doesn't, the test passes without checking what it intends to check.

A workaround *might* be to define the font to be used in the test, but I vaguely remember that also didn't fix the issue.

[Deleted User]

There are now WPT.fyi results for that test for Safari (not for GTK yet): https://wpt.fyi/results/html/canvas/element/manual/text/canvas.2d.lang.html?label=master&label=experimental&aligned

For Safari, the test fails as expected. To repeat it, that test passes locally with GTK.

[Deleted User]

For the record: text logged to stdout from WebKit, e.g. via `std::cout << "t" << std::endl;`, is not propagated to the terminal when running `./Tools/Scripts/run-webkit-tests`.

With `WebKitBuild/GTK/Debug/bin/WebKitTestRunner` it is.

[Deleted User]

`nextFont` [1] in `ComplexTextController` differs for the reftest vs opening the file in MiniBrowser:

Reftest: nextFont=Kochi Mincho
MiniBrowser: nextFont=Noto Sans CJK JP

[1] https://searchfox.org/wubkat/rev/9755214977d6a599c68289dfe5d601a811f1552c/Source/WebCore/platform/graphics/ComplexTextController.cpp#365

[Deleted User]

Locally installed  fonts:
```
$ fc-list | grep -i "Kochi\|Mincho\|Noto Sans CJK JP"
/usr/local/share/fonts/opentype/noto/NotoSansCJK-Regular.ttc: Noto Sans CJK JP:style=Regular
/usr/local/share/fonts/opentype/noto/NotoSansCJK-Bold.ttc: Noto Sans CJK JP:style=Bold
```

That is, apparently no Kochi Mincho.

It exists in the glib folder: https://searchfox.org/wubkat/search?q=Kochi&path=&case=false&regexp=false.

[Deleted User]

To summarize: WebKitTestRunner uses a specific set of fonts [1]. Only for glib/GTK/WPE. This should not be necessary. Instead WPTs should use `@font-face` if they need specific fonts. Given the latter isn't always the case, the environment, e.g. [2] should ensure the needed fonts are available.

[1] https://searchfox.org/wubkat/source/Tools/WebKitTestRunner/glib/fonts
[2] https://github.com/Igalia/webkit-container-sdk