Bug 178782

Summary: Flakiness dashboard: Make GTK and WPE builds inherit expectations from WK2
Product: WebKit Reporter: Alicia Boya García <aboya>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Severity: Normal CC: commit-queue, lforschler, mcatanzaro, webkit-bug-importer, zan
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Description Flags
Patch none

Description Alicia Boya García 2017-10-25 03:54:14 PDT
The following layout test is having an unexpected result on GTK according to the flakiness dashboard:



The interesting bit is that it should not because it's marked as expected [ Pass Failure Timeout ] in WK2 expectations file.

689:webkit.org/b/172452 http/tests/loading/resourceLoadStatistics/grandfathering.html [ Pass Failure Timeout ]

And GTK+ tests are run with those expectations in mind:

> run-webkit-tests --gtk http/tests/loading/resourceLoadStatistics/grandfathering.html 
Using port 'gtk-wk2'
Test configuration: <, x86, release>
Placing test results in /webkit/WebKitBuild/Release/layout-test-results
Baseline search path: platform/gtk -> platform/wk2 -> generic
Using Release build
Pixel tests disabled
Regular timeout: 15000, slow test timeout: 75000
Command line: /webkit/Tools/jhbuild/jhbuild-wrapper --gtk run /webkit/WebKitBuild/Release/bin/WebKitTestRunner -

Found 1 test; running 1, skipping 0.
Running 1 test

Running 1 WebKitTestRunner.     

The test ran as expected.                                          

Stopping HTTP server ...
Comment 1 Alicia Boya García 2017-12-12 10:45:10 PST
Turns out the flakiness dashboard is assuming GTK ports are WK1, therefore WK2 expectations are not taken into account.


Look at the top: Expectations for GTK_LINUX_WK1-RELEASE

This is the result of a logic in determineBuilderPlatform() that determines whether a platform is WK2 or WK1 based on its name: If it does not contain WK2, it assumes it's WK1.
Comment 2 Alicia Boya García 2017-12-12 11:34:56 PST
Created attachment 329132 [details]
Comment 3 WebKit Commit Bot 2017-12-12 16:11:50 PST
Comment on attachment 329132 [details]

Clearing flags on attachment: 329132

Committed r225819: <https://trac.webkit.org/changeset/225819>
Comment 4 WebKit Commit Bot 2017-12-12 16:11:51 PST
All reviewed patches have been landed.  Closing bug.
Comment 5 Radar WebKit Bug Importer 2017-12-12 16:12:26 PST
Comment 6 Alicia Boya García 2017-12-13 03:11:37 PST
The hosted version continues to run an old revision:


Note it still says WK1. Who should I ping for fixing that?
Comment 7 Michael Catanzaro 2017-12-13 08:57:16 PST
Try asking admin@webkit.org