webkitpy: Support reporting results for uncommon test configurations

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Jonathan Bedard]

It's possible to run layout tests is a variety of exotic ways. Rather than try and categorize each method as a separate style (as we have done for GuardMalloc and ASan), we should just allow buildbot to specifically override the default 'flavor' of a test result, since the 'flavor' option was designed for exactly such exotic configurations.

[Radar WebKit Bug Importer]

<rdar://problem/51524128>

[Jonathan Bedard]

Created attachment 371593 [details]
Patch

[Jonathan Bedard]

Created attachment 371607 [details]
Patch

[Aakash Jain]

Comment on attachment 371607 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=371607&action=review

> Tools/ChangeLog:3
> +        webkitpy: Support reporting results for exotic test configurations

exotic or uncommon/rare?

> Tools/Scripts/webkitpy/layout_tests/controllers/manager.py:271
> +            if not configuration.get('flavor', None):  # The --result-report-flavor argument should override wk1/wk2

flavor or result-report-flavor?

does this file already have concept of flavour? What does that do?

> Tools/Scripts/webkitpy/test/main.py:281
> +            print json.dumps(upload.configuration)

Is this for debugging or for production?

[Jonathan Bedard]

Comment on attachment 371607 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=371607&action=review

>> Tools/Scripts/webkitpy/layout_tests/controllers/manager.py:271
>> +            if not configuration.get('flavor', None):  # The --result-report-flavor argument should override wk1/wk2
> 
> flavor or result-report-flavor?
> 
> does this file already have concept of flavour? What does that do?

Here, flavor is correct because we're updating the upload configuration dictionary.

Layout tests already use this key to differentiate wk1 and wk2. More generally, the 'flavor' option is used as a wild-card key to distinguish testing configurations that don't fit into other more structured configuration options (like platform, style, model, ect.)

[Jonathan Bedard]

Created attachment 371611 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 371611 [details]
Patch

Clearing flags on attachment: 371611

Committed r246221: <https://trac.webkit.org/changeset/246221>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.