| Summary: | The results of A/B testing should state statistical significance | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | WebKit | Reporter: | Ryosuke Niwa <rniwa> | ||||||||
| Component: | WebKit Website | Assignee: | Ryosuke Niwa <rniwa> | ||||||||
| Status: | RESOLVED FIXED | ||||||||||
| Severity: | Normal | CC: | barraclough, benjamin, cdumez, kling, koivisto | ||||||||
| Priority: | P2 | ||||||||||
| Version: | 528+ (Nightly build) | ||||||||||
| Hardware: | Unspecified | ||||||||||
| OS: | Unspecified | ||||||||||
| Bug Depends on: | |||||||||||
| Bug Blocks: | 141443 | ||||||||||
| Attachments: |
|
||||||||||
|
Description
Ryosuke Niwa
2015-04-08 20:42:55 PDT
Created attachment 250410 [details]
Adds the feature
Created attachment 250411 [details]
Screenshot 1 (statistically significant)
Created attachment 250412 [details]
Screenshot 2 (statistically insignificant)
Comment on attachment 250410 [details] Adds the feature View in context: https://bugs.webkit.org/attachment.cgi?id=250410&action=review rs=me > Websites/perf.webkit.org/public/v2/app.js:1339 > + for (var i = 0; i < configurations.length; i++) { Looks like we only need to go up to configurations.length-1 ? > Websites/perf.webkit.org/public/v2/app.js:1341 > + var summary1 = configurations[i].summary; should probably be outside this for loop as it doesn't change. > Websites/perf.webkit.org/public/v2/app.js:1362 > + return 'Not statistically significant' + details; missing space after "significant". > Websites/perf.webkit.org/public/v2/js/statistics.js:67 > + return [null, null]; In the regular case you return a dict but here you are returning an array? Comment on attachment 250410 [details] Adds the feature View in context: https://bugs.webkit.org/attachment.cgi?id=250410&action=review Thanks for the review & nice bug catches! >> Websites/perf.webkit.org/public/v2/app.js:1339 >> + for (var i = 0; i < configurations.length; i++) { > > Looks like we only need to go up to configurations.length-1 ? Oops, nice catch! >> Websites/perf.webkit.org/public/v2/app.js:1341 >> + var summary1 = configurations[i].summary; > > should probably be outside this for loop as it doesn't change. Fixed. >> Websites/perf.webkit.org/public/v2/app.js:1362 >> + return 'Not statistically significant' + details; > > missing space after "significant". Oh, there is a leading space in "details". >> Websites/perf.webkit.org/public/v2/js/statistics.js:67 >> + return [null, null]; > > In the regular case you return a dict but here you are returning an array? Oops, fixed. Committed r182587: <http://trac.webkit.org/changeset/182587> |