Bug 32804 - SunSpider: crypto-aes has a call to Date.getTime() which makes it non-deterministic
Summary: SunSpider: crypto-aes has a call to Date.getTime() which makes it non-determi...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: PC OS X 10.5
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks: 43253
  Show dependency tree / graph
 
Reported: 2009-12-20 17:04 PST by Nicholas Nethercote
Modified: 2010-08-05 05:57 PDT (History)
4 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Nicholas Nethercote 2009-12-20 17:04:50 PST 
This is true as of SunSpider-0.9.1, but has been true for longer than that, AFAICT.  IMHO a benchmark suite should be deterministic.  The variation caused by the Date.getTime() is large enough that it changes the amount of code is traced by TraceMonkey.  I'd suggest hard-wiring a particular date+time into the test.

There may be some other non-determinism in SunSpider.  string-base64 and string-validate-input both call Math.random(), for example.

Thanks for introducing version numbers for SunSpider, BTW, I think that's an excellent decision.
Comment 1 Maciej Stachowiak 2009-12-20 23:36:26 PST 
Thanks for the report. 1.9.1 is not released yet so we may still be able to fix this, but if not, we'll probably do so for a later version.
Comment 2 Maciej Stachowiak 2010-02-28 14:26:43 PST 
I think we'll probably reserve fixing the nondeterminism for SunSpider 2.0.