Bug 178182 - REGRESSION(r223113): Reintroduced 20% regression on Kraken (Requested by rniwa on #webkit).
Summary: REGRESSION(r223113): Reintroduced 20% regression on Kraken (Requested by rniw...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: WebKit Commit Bot
URL:
Keywords: InRadar
Depends on:
Blocks: 177586 178050
  Show dependency treegraph
 
Reported: 2017-10-11 12:31 PDT by WebKit Commit Bot
Modified: 2017-10-12 08:46 PDT (History)
4 users (show)

See Also:


Attachments
ROLLOUT of r223113 (48.21 KB, patch)
2017-10-11 12:31 PDT, WebKit Commit Bot
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description WebKit Commit Bot 2017-10-11 12:31:18 PDT
https://trac.webkit.org/changeset/223113 broke the build:
Reintroduced 20% regression on Kraken (Requested by rniwa on #webkit).

This is an automatic bug report generated by webkitbot. If this bug
report was created because of a flaky test, please file a bug for the flaky
test (if we don't already have one on file) and dup this bug against that bug
so that we can track how often these flaky tests fail.
Comment 1 WebKit Commit Bot 2017-10-11 12:31:32 PDT
Created attachment 323447 [details]
ROLLOUT of r223113

Any committer can land this patch automatically by marking it commit-queue+.  The commit-queue will build and test the patch before landing to ensure that the rollout will be successful.  This process takes approximately 15 minutes.

If you would like to land the rollout faster, you can use the following command:

  webkit-patch land-attachment ATTACHMENT_ID

where ATTACHMENT_ID is the ID of this attachment.
Comment 2 WebKit Commit Bot 2017-10-11 12:33:52 PDT
Comment on attachment 323447 [details]
ROLLOUT of r223113

Clearing flags on attachment: 323447

Committed r223202: <https://trac.webkit.org/changeset/223202>
Comment 3 WebKit Commit Bot 2017-10-11 12:33:54 PDT
All reviewed patches have been landed.  Closing bug.
Comment 4 Filip Pizlo 2017-10-11 12:56:35 PDT
Was this initiated because of Kraken or because of Kraken and other bernchmarks?
Comment 5 Radar WebKit Bug Importer 2017-10-11 12:58:22 PDT
<rdar://problem/34939565>
Comment 6 Ryosuke Niwa 2017-10-11 13:01:14 PDT
(In reply to Filip Pizlo from comment #4)
> Was this initiated because of Kraken or because of Kraken and other
> bernchmarks?

When this patch got landed, 20% Kraken regression came up back on bots, and at the same time, we also observed Speedometer on iOS to regress by 2-4%. We've confirmed Speedometer to progress by the same amount when the initial patch got rolled out and regress again when the patch got re-landed.
Comment 7 Filip Pizlo 2017-10-11 13:34:41 PDT
(In reply to Ryosuke Niwa from comment #6)
> (In reply to Filip Pizlo from comment #4)
> > Was this initiated because of Kraken or because of Kraken and other
> > bernchmarks?
> 
> When this patch got landed, 20% Kraken regression came up back on bots,

You mean one bot.

> and
> at the same time, we also observed Speedometer on iOS to regress by 2-4%.
> We've confirmed Speedometer to progress by the same amount when the initial
> patch got rolled out and regress again when the patch got re-landed.

Kraken is not really a big focus, and it's not correct to say that we have a 20% regression without precisely qualifying that in fact, most iOS devices see no regression from that change.  I hope you agree with me that if you tell someone that something is X% regressed when in reality it is only X% regressed on one device and 0% regressed everywhere else, that this creates an unnecessary sense of urgency.

On the other hand, I agree with you the Speedometer is important and if it's true that all Speedo bots saw 2-4% regressions, then we should roll this out.

But if Speedometer is like Kraken in the sense that some bots see this and some don't then I'm not sure that an immediate rollout is super productive.
Comment 8 Ryosuke Niwa 2017-10-11 14:43:38 PDT
(In reply to Filip Pizlo from comment #7)
> (In reply to Ryosuke Niwa from comment #6)
> > (In reply to Filip Pizlo from comment #4)
> > > Was this initiated because of Kraken or because of Kraken and other
> > > bernchmarks?
> > 
> > When this patch got landed, 20% Kraken regression came up back on bots,
> 
> You mean one bot.
> 
> > and
> > at the same time, we also observed Speedometer on iOS to regress by 2-4%.
> > We've confirmed Speedometer to progress by the same amount when the initial
> > patch got rolled out and regress again when the patch got re-landed.
> 
> Kraken is not really a big focus, and it's not correct to say that we have a
> 20% regression without precisely qualifying that in fact, most iOS devices
> see no regression from that change.  I hope you agree with me that if you
> tell someone that something is X% regressed when in reality it is only X%
> regressed on one device and 0% regressed everywhere else, that this creates
> an unnecessary sense of urgency.

Right, I agree with you there.

> On the other hand, I agree with you the Speedometer is important and if it's
> true that all Speedo bots saw 2-4% regressions, then we should roll this out.
> 
> But if Speedometer is like Kraken in the sense that some bots see this and
> some don't then I'm not sure that an immediate rollout is super productive.

We saw on multiple bots. I'll send you links by an email.
Comment 9 Filip Pizlo 2017-10-12 08:46:04 PDT
(In reply to Ryosuke Niwa from comment #8)
> (In reply to Filip Pizlo from comment #7)
> > (In reply to Ryosuke Niwa from comment #6)
> > > (In reply to Filip Pizlo from comment #4)
> > > > Was this initiated because of Kraken or because of Kraken and other
> > > > bernchmarks?
> > > 
> > > When this patch got landed, 20% Kraken regression came up back on bots,
> > 
> > You mean one bot.
> > 
> > > and
> > > at the same time, we also observed Speedometer on iOS to regress by 2-4%.
> > > We've confirmed Speedometer to progress by the same amount when the initial
> > > patch got rolled out and regress again when the patch got re-landed.
> > 
> > Kraken is not really a big focus, and it's not correct to say that we have a
> > 20% regression without precisely qualifying that in fact, most iOS devices
> > see no regression from that change.  I hope you agree with me that if you
> > tell someone that something is X% regressed when in reality it is only X%
> > regressed on one device and 0% regressed everywhere else, that this creates
> > an unnecessary sense of urgency.
> 
> Right, I agree with you there.
> 
> > On the other hand, I agree with you the Speedometer is important and if it's
> > true that all Speedo bots saw 2-4% regressions, then we should roll this out.
> > 
> > But if Speedometer is like Kraken in the sense that some bots see this and
> > some don't then I'm not sure that an immediate rollout is super productive.
> 
> We saw on multiple bots. I'll send you links by an email.

We discussed over e-mail.  I'm going to roll this back in.