In the standalone shell version of Sunspider, the timer is started, then load() is called on the script, and finally the timer is stopped. load() reads the file from disk, and the IO effect of this leads to a 35% increase in variance (see https://bugzilla.mozilla.org/show_bug.cgi?id=580532#c24). The simplest solution is to read the file into a string, then start the timer, eval the string, and stop the timer. I'm told most JS engines have a read() builtin function for this (though on Firefox it is called snarf() for some reason).
Created attachment 63759 [details] Use eval() and read() instead of load() Use read() and eval() instead of load(). Currently, only v8 supports read(), but I'm adding it to SpiderMonkey, and it is a simple change to jsc, as far as I can tell.
This is indirectly fixed here: 2010-11-29 Geoffrey Garen <ggaren@apple.com> Improved accuracy of command-line SunSpider. Since run() loads the file before the timer is started, there's no need for this patch.