Bug 232553

Summary: Add a run-webkit-tests mode to A/B test a given feature
Product: WebKit Reporter: Tim Horton <thorton>
Component: New BugsAssignee: Tim Horton <thorton>
Status: RESOLVED FIXED    
Severity: Normal CC: ap, dino, ews-watchlist, glenn, heycam, jbedard, simon.fraser, webkit-bug-importer, wenson_hsieh
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch
none
Patch none

Description Tim Horton 2021-10-31 21:52:09 PDT
Add a run-webkit-tests mode to A/B test a given feature
Comment 1 Tim Horton 2021-10-31 21:52:31 PDT
Created attachment 442954 [details]
Patch
Comment 2 Tim Horton 2021-10-31 21:54:11 PDT
I am super open to naming adjustments (I've solicited suggestions elsewhere, hopefully the patch helps people come up with something), I don't love the name as-is.
Comment 3 Jonathan Bedard 2021-11-01 08:33:01 PDT
(In reply to Tim Horton from comment #2)
> I am super open to naming adjustments (I've solicited suggestions elsewhere,
> hopefully the patch helps people come up with something), I don't love the
> name as-is.

The name is the only thing I don't love....just some brainstormed options:

--compare-feature
--ab-test-feature
--feature-heade

I guess the problem I see is that the flag name is so long, maybe we keep the long name but also add a short one (maybe even just --ab or --feature) as an alternative
Comment 4 Tim Horton 2021-11-01 08:54:04 PDT
I think the tricky part is you want the long name to cover both the fact that it is running the tests in a weird way AND explain what the value of the argument should be (our test header format), thus the length. I did originally have `--ab-header`, but I'm not sure `ab` very well conveys `A/B` :) Maybe it does!
Comment 5 Tim Horton 2021-11-01 12:28:42 PDT
Created attachment 443000 [details]
Patch
Comment 6 EWS 2021-11-01 14:30:46 PDT
Committed r285132 (243771@main): <https://commits.webkit.org/243771@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 443000 [details].
Comment 7 Radar WebKit Bug Importer 2021-11-01 14:31:28 PDT
<rdar://problem/84899542>