Bug 198662

Summary: webkitpy: Support reporting results for uncommon test configurations
Product: WebKit Reporter: Jonathan Bedard <jbedard>
Component: Tools / TestsAssignee: Jonathan Bedard <jbedard>
Status: RESOLVED FIXED    
Severity: Normal CC: aakash_jain, commit-queue, ews, glenn, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch
none
Patch
none
Patch none

Description Jonathan Bedard 2019-06-07 09:23:09 PDT
It's possible to run layout tests is a variety of exotic ways. Rather than try and categorize each method as a separate style (as we have done for GuardMalloc and ASan), we should just allow buildbot to specifically override the default 'flavor' of a test result, since the 'flavor' option was designed for exactly such exotic configurations.
Comment 1 Radar WebKit Bug Importer 2019-06-07 09:24:31 PDT
<rdar://problem/51524128>
Comment 2 Jonathan Bedard 2019-06-07 10:30:16 PDT
Created attachment 371593 [details]
Patch
Comment 3 Jonathan Bedard 2019-06-07 13:43:22 PDT
Created attachment 371607 [details]
Patch
Comment 4 Aakash Jain 2019-06-07 14:28:23 PDT
Comment on attachment 371607 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=371607&action=review

> Tools/ChangeLog:3
> +        webkitpy: Support reporting results for exotic test configurations

exotic or uncommon/rare?

> Tools/Scripts/webkitpy/layout_tests/controllers/manager.py:271
> +            if not configuration.get('flavor', None):  # The --result-report-flavor argument should override wk1/wk2

flavor or result-report-flavor?

does this file already have concept of flavour? What does that do?

> Tools/Scripts/webkitpy/test/main.py:281
> +            print json.dumps(upload.configuration)

Is this for debugging or for production?
Comment 5 Jonathan Bedard 2019-06-07 14:37:52 PDT
Comment on attachment 371607 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=371607&action=review

>> Tools/Scripts/webkitpy/layout_tests/controllers/manager.py:271
>> +            if not configuration.get('flavor', None):  # The --result-report-flavor argument should override wk1/wk2
> 
> flavor or result-report-flavor?
> 
> does this file already have concept of flavour? What does that do?

Here, flavor is correct because we're updating the upload configuration dictionary.

Layout tests already use this key to differentiate wk1 and wk2. More generally, the 'flavor' option is used as a wild-card key to distinguish testing configurations that don't fit into other more structured configuration options (like platform, style, model, ect.)
Comment 6 Jonathan Bedard 2019-06-07 14:41:01 PDT
Created attachment 371611 [details]
Patch
Comment 7 WebKit Commit Bot 2019-06-07 15:50:16 PDT
Comment on attachment 371611 [details]
Patch

Clearing flags on attachment: 371611

Committed r246221: <https://trac.webkit.org/changeset/246221>
Comment 8 WebKit Commit Bot 2019-06-07 15:50:18 PDT
All reviewed patches have been landed.  Closing bug.