Bug 10958 - Buildbots give unreproducible results on layout tests
Summary: Buildbots give unreproducible results on layout tests
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 420+
Hardware: Mac OS X 10.4
: P2 Normal
Assignee: Nobody
URL: http://build.webkit.org/results/post-...
Keywords:
Depends on: 10923 11222
Blocks:
  Show dependency treegraph
 
Reported: 2006-09-20 17:43 PDT by Adam Roben (:aroben)
Modified: 2006-10-18 20:20 PDT (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Adam Roben (:aroben) 2006-09-20 17:43:01 PDT
The diff in the attached URL (and similar ones on a handful of other layout tests) only appear on the buildbots. So far no one has been able to reproduce these failures locally.
Comment 1 Adam Roben (:aroben) 2006-09-20 23:49:37 PDT
Here's another diff we can't reproduce:

http://build.webkit.org/results/post-commit-powerpc-mac-os-x/3386/fast/js/window-properties-diffs.txt

Oddly, I did reproduce this once after building on Leopard, but as the buildbots are all on Tiger, this clearly requires further investigation.
Comment 2 Adam Roben (:aroben) 2006-09-21 00:01:39 PDT
Now I'm back on Tiger with a ToT build, and I can get the window-properties test to show the same results as the buildbot only if I run all the layout tests. Running just that one test or even the whole fast/js directory will generate the checked in results (i.e. the test will pass), but running all the layout tests at once will generate the (failed) results that the buildbot gets.
Comment 3 Adele Peterson 2006-09-21 01:11:41 PDT
I can confirm what Adam's seeing.  I'm on Tiger, and I see the window-properties test only fail when run with the whole suite of tests.
Comment 4 Adam Roben (:aroben) 2006-09-21 01:16:13 PDT
Opened bug 10965 to track the problem with window-properties, since it seems to be a separate issue. This bug can continue to function as a meta-bug tracking the larger issue of the buildbots displaying unreproducible behavior.
Comment 5 Adam Roben (:aroben) 2006-09-21 09:00:06 PDT
Setting blocked by bug 10923, which lists the specific tests that the buildbots show the unreproducible "frame * scrolled to *" diff
Comment 6 Adam Roben (:aroben) 2006-10-08 13:38:17 PDT
New pieces of evidence:

fast/js/window-properties now seems to intermittently report that window.document.location.replace is a function.

http://build.webkit.org/results/post-commit-powerpc-mac-os-x/3700/results.html and http://build.webkit.org/results/post-commit-powerpc-mac-os-x/3703/results.html show this, but http://build.webkit.org/results/post-commit-powerpc-mac-os-x/3701/results.html and http://build.webkit.org/results/post-commit-powerpc-mac-os-x/3702/results.html don't.
Comment 7 Geoffrey Garen 2006-10-18 16:16:30 PDT
I don't think this umbrella bug is helpful anymore. Each failure has its own bug, and there doesn't seem to be a link between them.
Comment 8 Adam Roben (:aroben) 2006-10-18 20:20:07 PDT
J'agree with ggaren