Bug 174304 - A/B testing results page show results for the top-level tests instead of the one being analyzed
Summary: A/B testing results page show results for the top-level tests instead of the ...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Perf Dashboard (show other bugs)
Version: Safari Technology Preview
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Ryosuke Niwa
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-07-09 18:54 PDT by Ryosuke Niwa
Modified: 2017-07-10 20:45 PDT (History)
6 users (show)

See Also:


Attachments
Fixes the bug (5.77 KB, patch)
2017-07-09 19:04 PDT, Ryosuke Niwa
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ryosuke Niwa 2017-07-09 18:54:37 PDT
After the perf try bots support was added, the results shown right above the revision table
always show the results for the top-level tests instead of the one being biesected.

e.g. If we're bisecting MotionMark's Images subtest, we'd like to see how A/B compares in Images subtest,
not the overall MotionMark's score, which may or may not show the actual regression / progression
due to the noise from other tests.

What's even more confusing is the fact the stacking graph charts DO show the progression/regression
amount based on the subtest.
Comment 1 Ryosuke Niwa 2017-07-09 19:04:20 PDT
Created attachment 314959 [details]
Fixes the bug
Comment 2 Antti Koivisto 2017-07-10 01:18:21 PDT
Comment on attachment 314959 [details]
Fixes the bug

View in context: https://bugs.webkit.org/attachment.cgi?id=314959&action=review

> Websites/perf.webkit.org/public/v3/models/analysis-results.js:23
> +        for (let metric of test.metrics()) {

const?
Comment 3 Ryosuke Niwa 2017-07-10 20:45:01 PDT
Comment on attachment 314959 [details]
Fixes the bug

Clearing flags on attachment: 314959

Committed r219321: <http://trac.webkit.org/changeset/219321>
Comment 4 Ryosuke Niwa 2017-07-10 20:45:03 PDT
All reviewed patches have been landed.  Closing bug.