Bug 32804

Summary:	SunSpider: crypto-aes has a call to Date.getTime() which makes it non-deterministic
Product:	WebKit	Reporter:	Nicholas Nethercote <nnethercote>
Component:	Tools / Tests	Assignee:	Nobody <webkit-unassigned>
Status:	NEW ---
Severity:	Normal	CC:	jwalden+bwo, mjs, sdwilsh, seth.gaurav
Priority:	P2
Version:	528+ (Nightly build)
Hardware:	PC
OS:	OS X 10.5
Bug Depends on:
Bug Blocks:	43253

Description Nicholas Nethercote 2009-12-20 17:04:50 PST

This is true as of SunSpider-0.9.1, but has been true for longer than that, AFAICT.  IMHO a benchmark suite should be deterministic.  The variation caused by the Date.getTime() is large enough that it changes the amount of code is traced by TraceMonkey.  I'd suggest hard-wiring a particular date+time into the test.

There may be some other non-determinism in SunSpider.  string-base64 and string-validate-input both call Math.random(), for example.

Thanks for introducing version numbers for SunSpider, BTW, I think that's an excellent decision.

Comment 1 Maciej Stachowiak 2009-12-20 23:36:26 PST

Thanks for the report. 1.9.1 is not released yet so we may still be able to fix this, but if not, we'll probably do so for a later version.

Comment 2 Maciej Stachowiak 2010-02-28 14:26:43 PST

I think we'll probably reserve fixing the nondeterminism for SunSpider 2.0.