|Summary:||Flakiness dashboard: Make GTK and WPE builds inherit expectations from WK2|
|Product:||WebKit||Reporter:||Alicia Boya García <aboya>|
|Component:||Tools / Tests||Assignee:||Nobody <webkit-unassigned>|
|Severity:||Normal||CC:||commit-queue, lforschler, mcatanzaro, webkit-bug-importer, zan|
|Version:||WebKit Nightly Build|
Description Alicia Boya García 2017-10-25 03:54:14 PDT
The following layout test is having an unexpected result on GTK according to the flakiness dashboard: http/tests/loading/resourceLoadStatistics/grandfathering.html https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Floading%2FresourceLoadStatistics%2Fgrandfathering.html The interesting bit is that it should not because it's marked as expected [ Pass Failure Timeout ] in WK2 expectations file. platform/wk2/TestExpectations 689:webkit.org/b/172452 http/tests/loading/resourceLoadStatistics/grandfathering.html [ Pass Failure Timeout ] And GTK+ tests are run with those expectations in mind: > run-webkit-tests --gtk http/tests/loading/resourceLoadStatistics/grandfathering.html Using port 'gtk-wk2' Test configuration: <, x86, release> Placing test results in /webkit/WebKitBuild/Release/layout-test-results Baseline search path: platform/gtk -> platform/wk2 -> generic Using Release build Pixel tests disabled Regular timeout: 15000, slow test timeout: 75000 Command line: /webkit/Tools/jhbuild/jhbuild-wrapper --gtk run /webkit/WebKitBuild/Release/bin/WebKitTestRunner - Found 1 test; running 1, skipping 0. Running 1 test Running 1 WebKitTestRunner. The test ran as expected. Stopping HTTP server ...
Comment 1 Alicia Boya García 2017-12-12 10:45:10 PST
Turns out the flakiness dashboard is assuming GTK ports are WK1, therefore WK2 expectations are not taken into account. https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showWrongExpectations=true&builder=GTK%20Linux%2064-bit%20Release%20(Tests) Look at the top: Expectations for GTK_LINUX_WK1-RELEASE This is the result of a logic in determineBuilderPlatform() that determines whether a platform is WK2 or WK1 based on its name: If it does not contain WK2, it assumes it's WK1.
Comment 3 WebKit Commit Bot 2017-12-12 16:11:50 PST
Comment on attachment 329132 [details] Patch Clearing flags on attachment: 329132 Committed r225819: <https://trac.webkit.org/changeset/225819>
Comment 4 WebKit Commit Bot 2017-12-12 16:11:51 PST
All reviewed patches have been landed. Closing bug.
Comment 6 Alicia Boya García 2017-12-13 03:11:37 PST
The hosted version continues to run an old revision: https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#builder=GTK%20Linux%2064-bit%20Release%20(Tests) Note it still says WK1. Who should I ping for fixing that?