Bug 32804

Summary: SunSpider: crypto-aes has a call to Date.getTime() which makes it non-deterministic
Product: WebKit Reporter: Nicholas Nethercote <nnethercote>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: NEW ---    
Severity: Normal CC: jwalden+bwo, mjs, sdwilsh, seth.gaurav
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: PC   
OS: OS X 10.5   
Bug Depends on:    
Bug Blocks: 43253    

Description Nicholas Nethercote 2009-12-20 17:04:50 PST
This is true as of SunSpider-0.9.1, but has been true for longer than that, AFAICT.  IMHO a benchmark suite should be deterministic.  The variation caused by the Date.getTime() is large enough that it changes the amount of code is traced by TraceMonkey.  I'd suggest hard-wiring a particular date+time into the test.

There may be some other non-determinism in SunSpider.  string-base64 and string-validate-input both call Math.random(), for example.

Thanks for introducing version numbers for SunSpider, BTW, I think that's an excellent decision.
Comment 1 Maciej Stachowiak 2009-12-20 23:36:26 PST
Thanks for the report. 1.9.1 is not released yet so we may still be able to fix this, but if not, we'll probably do so for a later version.
Comment 2 Maciej Stachowiak 2010-02-28 14:26:43 PST
I think we'll probably reserve fixing the nondeterminism for SunSpider 2.0.