Bug 178782 – Flakiness dashboard: Make GTK and WPE builds inherit expectations from WK2

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Alicia Boya García]

The following layout test is having an unexpected result on GTK according to the flakiness dashboard:

http/tests/loading/resourceLoadStatistics/grandfathering.html

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Floading%2FresourceLoadStatistics%2Fgrandfathering.html

The interesting bit is that it should not because it's marked as expected [ Pass Failure Timeout ] in WK2 expectations file.

platform/wk2/TestExpectations
689:webkit.org/b/172452 http/tests/loading/resourceLoadStatistics/grandfathering.html [ Pass Failure Timeout ]

And GTK+ tests are run with those expectations in mind:

> run-webkit-tests --gtk http/tests/loading/resourceLoadStatistics/grandfathering.html 
Using port 'gtk-wk2'
Test configuration: <, x86, release>
Placing test results in /webkit/WebKitBuild/Release/layout-test-results
Baseline search path: platform/gtk -> platform/wk2 -> generic
Using Release build
Pixel tests disabled
Regular timeout: 15000, slow test timeout: 75000
Command line: /webkit/Tools/jhbuild/jhbuild-wrapper --gtk run /webkit/WebKitBuild/Release/bin/WebKitTestRunner -

Found 1 test; running 1, skipping 0.
                        
Running 1 test

Running 1 WebKitTestRunner.     

The test ran as expected.                                          

Stopping HTTP server ...

[Alicia Boya García]

Turns out the flakiness dashboard is assuming GTK ports are WK1, therefore WK2 expectations are not taken into account.

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showWrongExpectations=true&builder=GTK%20Linux%2064-bit%20Release%20(Tests)

Look at the top: Expectations for GTK_LINUX_WK1-RELEASE

This is the result of a logic in determineBuilderPlatform() that determines whether a platform is WK2 or WK1 based on its name: If it does not contain WK2, it assumes it's WK1.

[Alicia Boya García]

Created attachment 329132 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 329132 [details]
Patch

Clearing flags on attachment: 329132

Committed r225819: <https://trac.webkit.org/changeset/225819>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/36008110>

[Alicia Boya García]

The hosted version continues to run an old revision:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#builder=GTK%20Linux%2064-bit%20Release%20(Tests)

Note it still says WK1. Who should I ping for fixing that?

[Michael Catanzaro]

Try asking [e-mail]