Bug 178782 - Flakiness dashboard: Make GTK and WPE builds inherit expectations from WK2
Summary: Flakiness dashboard: Make GTK and WPE builds inherit expectations from WK2
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
Keywords: InRadar
Depends on:
Reported: 2017-10-25 03:54 PDT by Alicia Boya García
Modified: 2017-12-13 08:57 PST (History)
5 users (show)

See Also:

Patch (2.88 KB, patch)
2017-12-12 11:34 PST, Alicia Boya García
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Alicia Boya García 2017-10-25 03:54:14 PDT
The following layout test is having an unexpected result on GTK according to the flakiness dashboard:



The interesting bit is that it should not because it's marked as expected [ Pass Failure Timeout ] in WK2 expectations file.

689:webkit.org/b/172452 http/tests/loading/resourceLoadStatistics/grandfathering.html [ Pass Failure Timeout ]

And GTK+ tests are run with those expectations in mind:

> run-webkit-tests --gtk http/tests/loading/resourceLoadStatistics/grandfathering.html 
Using port 'gtk-wk2'
Test configuration: <, x86, release>
Placing test results in /webkit/WebKitBuild/Release/layout-test-results
Baseline search path: platform/gtk -> platform/wk2 -> generic
Using Release build
Pixel tests disabled
Regular timeout: 15000, slow test timeout: 75000
Command line: /webkit/Tools/jhbuild/jhbuild-wrapper --gtk run /webkit/WebKitBuild/Release/bin/WebKitTestRunner -

Found 1 test; running 1, skipping 0.
Running 1 test

Running 1 WebKitTestRunner.     

The test ran as expected.                                          

Stopping HTTP server ...
Comment 1 Alicia Boya García 2017-12-12 10:45:10 PST
Turns out the flakiness dashboard is assuming GTK ports are WK1, therefore WK2 expectations are not taken into account.


Look at the top: Expectations for GTK_LINUX_WK1-RELEASE

This is the result of a logic in determineBuilderPlatform() that determines whether a platform is WK2 or WK1 based on its name: If it does not contain WK2, it assumes it's WK1.
Comment 2 Alicia Boya García 2017-12-12 11:34:56 PST
Created attachment 329132 [details]
Comment 3 WebKit Commit Bot 2017-12-12 16:11:50 PST
Comment on attachment 329132 [details]

Clearing flags on attachment: 329132

Committed r225819: <https://trac.webkit.org/changeset/225819>
Comment 4 WebKit Commit Bot 2017-12-12 16:11:51 PST
All reviewed patches have been landed.  Closing bug.
Comment 5 Radar WebKit Bug Importer 2017-12-12 16:12:26 PST
Comment 6 Alicia Boya García 2017-12-13 03:11:37 PST
The hosted version continues to run an old revision:


Note it still says WK1. Who should I ping for fixing that?
Comment 7 Michael Catanzaro 2017-12-13 08:57:16 PST
Try asking admin@webkit.org