32804 2009-12-20 17:04:50 -0800 SunSpider: crypto-aes has a call to Date.getTime() which makes it non-deterministic 2010-08-05 05:57:20 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) PC OS X 10.5 NEW P2 Normal --- 43253 1 nnethercote webkit-unassigned jwalden+bwo mjs sdwilsh seth.gaurav oldest_to_newest 173837 0 nnethercote 2009-12-20 17:04:50 -0800 This is true as of SunSpider-0.9.1, but has been true for longer than that, AFAICT. IMHO a benchmark suite should be deterministic. The variation caused by the Date.getTime() is large enough that it changes the amount of code is traced by TraceMonkey. I'd suggest hard-wiring a particular date+time into the test. There may be some other non-determinism in SunSpider. string-base64 and string-validate-input both call Math.random(), for example. Thanks for introducing version numbers for SunSpider, BTW, I think that's an excellent decision. 173896 1 mjs 2009-12-20 23:36:26 -0800 Thanks for the report. 1.9.1 is not released yet so we may still be able to fix this, but if not, we'll probably do so for a later version. 194606 2 mjs 2010-02-28 14:26:43 -0800 I think we'll probably reserve fixing the nondeterminism for SunSpider 2.0.