The diff in the attached URL (and similar ones on a handful of other layout tests) only appear on the buildbots. So far no one has been able to reproduce these failures locally.
Here's another diff we can't reproduce: http://build.webkit.org/results/post-commit-powerpc-mac-os-x/3386/fast/js/window-properties-diffs.txt Oddly, I did reproduce this once after building on Leopard, but as the buildbots are all on Tiger, this clearly requires further investigation.
Now I'm back on Tiger with a ToT build, and I can get the window-properties test to show the same results as the buildbot only if I run all the layout tests. Running just that one test or even the whole fast/js directory will generate the checked in results (i.e. the test will pass), but running all the layout tests at once will generate the (failed) results that the buildbot gets.
I can confirm what Adam's seeing. I'm on Tiger, and I see the window-properties test only fail when run with the whole suite of tests.
Opened bug 10965 to track the problem with window-properties, since it seems to be a separate issue. This bug can continue to function as a meta-bug tracking the larger issue of the buildbots displaying unreproducible behavior.
Setting blocked by bug 10923, which lists the specific tests that the buildbots show the unreproducible "frame * scrolled to *" diff
New pieces of evidence: fast/js/window-properties now seems to intermittently report that window.document.location.replace is a function. http://build.webkit.org/results/post-commit-powerpc-mac-os-x/3700/results.html and http://build.webkit.org/results/post-commit-powerpc-mac-os-x/3703/results.html show this, but http://build.webkit.org/results/post-commit-powerpc-mac-os-x/3701/results.html and http://build.webkit.org/results/post-commit-powerpc-mac-os-x/3702/results.html don't.
I don't think this umbrella bug is helpful anymore. Each failure has its own bug, and there doesn't seem to be a link between them.
J'agree with ggaren