Bug 10958
| Summary: | Buildbots give unreproducible results on layout tests | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Adam Roben (:aroben) <aroben> |
| Component: | Tools / Tests | Assignee: | Nobody <webkit-unassigned> |
| Status: | RESOLVED FIXED | ||
| Severity: | Normal | CC: | ap, ddkilzer, mitz, timothy |
| Priority: | P2 | ||
| Version: | 420+ | ||
| Hardware: | Mac | ||
| OS: | OS X 10.4 | ||
| URL: | http://build.webkit.org/results/post-commit-powerpc-mac-os-x/3383/fast/css/acid2-pixel-diffs.txt | ||
| Bug Depends on: | 10923, 11222 | ||
| Bug Blocks: | |||
Adam Roben (:aroben)
The diff in the attached URL (and similar ones on a handful of other layout tests) only appear on the buildbots. So far no one has been able to reproduce these failures locally.
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Adam Roben (:aroben)
Here's another diff we can't reproduce:
http://build.webkit.org/results/post-commit-powerpc-mac-os-x/3386/fast/js/window-properties-diffs.txt
Oddly, I did reproduce this once after building on Leopard, but as the buildbots are all on Tiger, this clearly requires further investigation.
Adam Roben (:aroben)
Now I'm back on Tiger with a ToT build, and I can get the window-properties test to show the same results as the buildbot only if I run all the layout tests. Running just that one test or even the whole fast/js directory will generate the checked in results (i.e. the test will pass), but running all the layout tests at once will generate the (failed) results that the buildbot gets.
Adele Peterson
I can confirm what Adam's seeing. I'm on Tiger, and I see the window-properties test only fail when run with the whole suite of tests.
Adam Roben (:aroben)
Opened bug 10965 to track the problem with window-properties, since it seems to be a separate issue. This bug can continue to function as a meta-bug tracking the larger issue of the buildbots displaying unreproducible behavior.
Adam Roben (:aroben)
Setting blocked by bug 10923, which lists the specific tests that the buildbots show the unreproducible "frame * scrolled to *" diff
Adam Roben (:aroben)
New pieces of evidence:
fast/js/window-properties now seems to intermittently report that window.document.location.replace is a function.
http://build.webkit.org/results/post-commit-powerpc-mac-os-x/3700/results.html and http://build.webkit.org/results/post-commit-powerpc-mac-os-x/3703/results.html show this, but http://build.webkit.org/results/post-commit-powerpc-mac-os-x/3701/results.html and http://build.webkit.org/results/post-commit-powerpc-mac-os-x/3702/results.html don't.
Geoffrey Garen
I don't think this umbrella bug is helpful anymore. Each failure has its own bug, and there doesn't seem to be a link between them.
Adam Roben (:aroben)
J'agree with ggaren