Bug 32804
| Summary: | SunSpider: crypto-aes has a call to Date.getTime() which makes it non-deterministic | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Nicholas Nethercote <nnethercote> |
| Component: | Tools / Tests | Assignee: | Nobody <webkit-unassigned> |
| Status: | NEW | ||
| Severity: | Normal | CC: | jwalden+bwo, mjs, sdwilsh, seth.gaurav |
| Priority: | P2 | ||
| Version: | 528+ (Nightly build) | ||
| Hardware: | PC | ||
| OS: | OS X 10.5 | ||
| Bug Depends on: | |||
| Bug Blocks: | 43253 | ||
Nicholas Nethercote
This is true as of SunSpider-0.9.1, but has been true for longer than that, AFAICT. IMHO a benchmark suite should be deterministic. The variation caused by the Date.getTime() is large enough that it changes the amount of code is traced by TraceMonkey. I'd suggest hard-wiring a particular date+time into the test.
There may be some other non-determinism in SunSpider. string-base64 and string-validate-input both call Math.random(), for example.
Thanks for introducing version numbers for SunSpider, BTW, I think that's an excellent decision.
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Maciej Stachowiak
Thanks for the report. 1.9.1 is not released yet so we may still be able to fix this, but if not, we'll probably do so for a later version.
Maciej Stachowiak
I think we'll probably reserve fixing the nondeterminism for SunSpider 2.0.