Bug 101782 – [v8?] 37% SunSpider regression

• Status: RESOLVED
• Resolution: WONTFIX
• Component: Tools / Tests

[Ojan Vafai]

It shows in some of the other sunspider tests as well, but here are the most clear graphs.

http://build.chromium.org/f/chromium/perf/chromium-rel-win7-webkit/sunspider/report.html?rev=166898&graph=math-spectral-norm&trace=t&history=150

http://build.chromium.org/f/chromium/perf/chromium-rel-win7-webkit/sunspider/report.html?rev=166898&graph=access-fannkuch&trace=t&history=150

http://build.chromium.org/f/chromium/perf/chromium-rel-win7-webkit/sunspider/report.html?rev=166898&graph=bitops-3bit-bits-in-byte&trace=t&history=150

It's clearly not a chromium side change. Chromium regression range: http://build.chromium.org/f/chromium/perf/dashboard/ui/changelog.html?url=/trunk/src&mode=html&range=166683:166688

WebKit regression range: http://trac.webkit.org/log/?verbose=on&rev=133904&stop_rev=133902

Looks pretty clearly to be http://trac.webkit.org/changeset/133904/, since we got the equivalent performance gain when http://trac.webkit.org/changeset/133429/ was committed.

It would be good to understand what http://trac.webkit.org/changeset/133429/ does better so we can understand:
a) Why it makes sunspider so much faster.
b) Why it made the other tests slower and use more memory.

[Ojan Vafai]

This is also probably V8 only, but I don't have an easy way to verify that.

[Ojan Vafai]

Also a 7% *improvement* on dom_perf CreateNodes: http://build.chromium.org/f/chromium/perf/linux-release-webkit-latest/dom_perf/report.html?rev=166857&graph=CreateNodes&trace=score&history=150

And a 20% regression on dom_perf GetElements:
http://build.chromium.org/f/chromium/perf/mac-release-10.6-webkit-latest/dom_perf/report.html?rev=166869&graph=Get%20Elements&trace=score&history=150

[Adam Barth]

Huh?  I don't understand how http://trac.webkit.org/changeset/133429/ could have affected SunSpider.

[Ojan Vafai]

Yeah, if you look at when the patch was landed and reverted though, it's clearly this revision. I think the v8 properties we're changing must have an effect we don't understand?

[Adam Barth]

Maybe the feature itself slows down sunspider and that patch just happened to turn it off?

[Dimitri Glazkov (Google)]

wat. This makes no sense :-\

[Ojan Vafai]

To be clear, these tests take ~2ms to run, so a 30% regression is relatively small. We should considering changing our sunspider harness on these bots to do runs/second so we can get more useful numbers.

I think someone needs to take a look at what the IDL changes affect in terms of the code generator output to make sense of this. Just a guess though.

[Kentaro Hara]

(In reply to comment #7)
> I think someone needs to take a look at what the IDL changes affect in terms of the code generator output to make sense of this. Just a guess though.

The IDL change just changed DOM attributes that have [Conditional=SHADOW_DOM]. So at the very least, it will affect behaviors where SHADOW_DOM is enabled. On the other hand, SunSpider is not yet using SHADOW_DOM, right?

[Hajime Morrita]

(In reply to comment #8)
> (In reply to comment #7)
> > I think someone needs to take a look at what the IDL changes affect in terms of the code generator output to make sense of this. Just a guess though.
> 
> The IDL change just changed DOM attributes that have [Conditional=SHADOW_DOM]. So at the very least, it will affect behaviors where SHADOW_DOM is enabled. On the other hand, SunSpider is not yet using SHADOW_DOM, right?

[Anders Carlsson]

V8 is gone.