Bug 232553 - Add a run-webkit-tests mode to A/B test a given feature
Summary: Add a run-webkit-tests mode to A/B test a given feature
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Tim Horton
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-10-31 21:52 PDT by Tim Horton
Modified: 2021-11-01 14:31 PDT (History)
9 users (show)

See Also:


Attachments
Patch (15.94 KB, patch)
2021-10-31 21:52 PDT, Tim Horton
no flags Details | Formatted Diff | Diff
Patch (15.75 KB, patch)
2021-11-01 12:28 PDT, Tim Horton
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tim Horton 2021-10-31 21:52:09 PDT
Add a run-webkit-tests mode to A/B test a given feature
Comment 1 Tim Horton 2021-10-31 21:52:31 PDT
Created attachment 442954 [details]
Patch
Comment 2 Tim Horton 2021-10-31 21:54:11 PDT
I am super open to naming adjustments (I've solicited suggestions elsewhere, hopefully the patch helps people come up with something), I don't love the name as-is.
Comment 3 Jonathan Bedard 2021-11-01 08:33:01 PDT
(In reply to Tim Horton from comment #2)
> I am super open to naming adjustments (I've solicited suggestions elsewhere,
> hopefully the patch helps people come up with something), I don't love the
> name as-is.

The name is the only thing I don't love....just some brainstormed options:

--compare-feature
--ab-test-feature
--feature-heade

I guess the problem I see is that the flag name is so long, maybe we keep the long name but also add a short one (maybe even just --ab or --feature) as an alternative
Comment 4 Tim Horton 2021-11-01 08:54:04 PDT
I think the tricky part is you want the long name to cover both the fact that it is running the tests in a weird way AND explain what the value of the argument should be (our test header format), thus the length. I did originally have `--ab-header`, but I'm not sure `ab` very well conveys `A/B` :) Maybe it does!
Comment 5 Tim Horton 2021-11-01 12:28:42 PDT
Created attachment 443000 [details]
Patch
Comment 6 EWS 2021-11-01 14:30:46 PDT
Committed r285132 (243771@main): <https://commits.webkit.org/243771@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 443000 [details].
Comment 7 Radar WebKit Bug Importer 2021-11-01 14:31:28 PDT
<rdar://problem/84899542>