|Summary:||webkitpy: Support reporting results for uncommon test configurations|
|Product:||WebKit||Reporter:||Jonathan Bedard <jbedard>|
|Component:||Tools / Tests||Assignee:||Jonathan Bedard <jbedard>|
|Severity:||Normal||CC:||aakash_jain, commit-queue, ews, glenn, webkit-bug-importer|
|Version:||WebKit Nightly Build|
Description Jonathan Bedard 2019-06-07 09:23:09 PDT
It's possible to run layout tests is a variety of exotic ways. Rather than try and categorize each method as a separate style (as we have done for GuardMalloc and ASan), we should just allow buildbot to specifically override the default 'flavor' of a test result, since the 'flavor' option was designed for exactly such exotic configurations.
Comment 4 Aakash Jain 2019-06-07 14:28:23 PDT
Comment on attachment 371607 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=371607&action=review > Tools/ChangeLog:3 > + webkitpy: Support reporting results for exotic test configurations exotic or uncommon/rare? > Tools/Scripts/webkitpy/layout_tests/controllers/manager.py:271 > + if not configuration.get('flavor', None): # The --result-report-flavor argument should override wk1/wk2 flavor or result-report-flavor? does this file already have concept of flavour? What does that do? > Tools/Scripts/webkitpy/test/main.py:281 > + print json.dumps(upload.configuration) Is this for debugging or for production?
Comment 5 Jonathan Bedard 2019-06-07 14:37:52 PDT
Comment on attachment 371607 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=371607&action=review >> Tools/Scripts/webkitpy/layout_tests/controllers/manager.py:271 >> + if not configuration.get('flavor', None): # The --result-report-flavor argument should override wk1/wk2 > > flavor or result-report-flavor? > > does this file already have concept of flavour? What does that do? Here, flavor is correct because we're updating the upload configuration dictionary. Layout tests already use this key to differentiate wk1 and wk2. More generally, the 'flavor' option is used as a wild-card key to distinguish testing configurations that don't fit into other more structured configuration options (like platform, style, model, ect.)
Comment 7 WebKit Commit Bot 2019-06-07 15:50:16 PDT
Comment on attachment 371611 [details] Patch Clearing flags on attachment: 371611 Committed r246221: <https://trac.webkit.org/changeset/246221>
Comment 8 WebKit Commit Bot 2019-06-07 15:50:18 PDT
All reviewed patches have been landed. Closing bug.