|
Description
WebKit Commit Bot
2011-04-12 21:47:33 PDT
Created attachment 89336 [details]
Failure diff from eseidel-cq-sf
On what platform is this bot running? oh, i see 10.6.4 mac. this test hasn't been flakey on the core snowleopard bots. Given that this is the first time we've seen it flake, it's likely a very new (or very rare) flake. I wouldn't worry about it unless we see it happen again. The flake (as you can tell from the attachment diff) was caused by the loader callbacks not showing up for some reason. It's possible they bled into a later test or something. The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 89425 [details] on bug 58284. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 89535 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 89415 [details] on bug 58460. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 89541 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 89451 [details] on bug 58475. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 89551 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 89229 [details] on bug 58188. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 89554 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 89458 [details] on bug 58477. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 89572 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 89518 [details] on bug 57818. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 89719 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 89558 [details] on bug 58447. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 89732 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 89621 [details] on bug 53556. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 89747 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 89639 [details] on bug 58566. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 89754 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 89771 [details] on bug 58651. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 89795 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 89669 [details] on bug 58573. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 89876 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 89949 [details] on bug 58742. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 89954 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 90131 [details] on bug 58859. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 90135 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 90098 [details] on bug 58346. Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.6 Created attachment 90139 [details]
Failure diff from cr-jail-7
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 90075 [details] on bug 58813. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 90143 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 90114 [details] on bug 58419. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 90236 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 90271 [details] on bug 57678. Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.6 Created attachment 90297 [details]
Failure diff from cr-jail-7
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 90152 [details] on bug 58865. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 90484 [details]
Failure diff from eseidel-cq-sf
The diff is the same every time. We're also no longer stopping after the first failure, so I can state for sure that when this has recently flaked, its been the only flake. Ha. Looking at the test it looks obviously flaky. It sets up the loader callbacks on onload. :) It's very possible that the favicon has already loaded by then! We just need to move the setup out of onload and into a <script> tag to fix this. Oh. Maybe the reload is supposed to trigger the callbacks to fire again... in which case it would not be obviously racy. Looking. The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 90447 [details] on bug 59040. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 90497 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 90496 [details] on bug 59076. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 90502 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 90523 [details] on bug 59097. Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.6 Created attachment 90537 [details]
Failure diff from cr-jail-7
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 90284 [details] on bug 55021. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 90543 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 90549 [details] on bug 58772. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 90571 [details]
Failure diff from eseidel-cq-sf
(In reply to comment #43) > The diff is the same every time. > > We're also no longer stopping after the first failure, so I can state for sure that when this has recently flaked, its been the only flake. > > Ha. Looking at the test it looks obviously flaky. It sets up the loader callbacks on onload. :) It's very possible that the favicon has already loaded by then! > > We just need to move the setup out of onload and into a <script> tag to fix this. Couple things worth mentioning: 1) this test is basically a clone of http://trac.webkit.org/browser/trunk/LayoutTests/http/tests/misc/favicon-loads-with-images-disabled.html. The only thing difference in this test is the preference override. 2) This test is only flaking on 2 particular commit-queue bots, mostly eseidel-cq-sf and sometimes cr-jail-7. The various core testers for Mac and Windows aren't failing this test. The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 90321 [details] on bug 58969. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4 Created attachment 90599 [details]
Failure diff from eseidel-cq-sf
I've updated eseidel-cq-sf to 10.6.7. I'm also now attempting to reproduce this failure locally on my work machine. If I run the test twice in a row it fails in exactly the way seen on the bots: run-webkit-tests http/tests/misc/favicon-loads-with-icon-loading-override.html http/tests/misc/favicon-loads-with-icon-loading-override.html I suspect that we don't show loader callbacks when we pull something from the cache. I suspect that sometimes the favicon gets into the cache before we start the test (by issuing a reload) and thus we never send callbacks for it. For reference, the other test which Alice mentioned does not fail when run twice in a row: run-webkit-tests http/tests/misc/favicon-loads-with-images-disabled.html http/tests/misc/favicon-loads-with-images-disabled.html (In reply to comment #55) > 2) This test is only flaking on 2 particular commit-queue bots, mostly eseidel-cq-sf and sometimes cr-jail-7. The various core testers for Mac and Windows aren't failing this test. There are many tests which fail on the real bots that we just don't notice because they don't fail often enough for someone to see (and file a bug on). Because the commit-queues always log flakes to these bugs we are better able to track flakes on the cq. That said, it's very possible this is a cq-only failure. However being able to reproduce the failure on my work machine makes me think this is probably flaky everywhere (but only when the ico happens to load before the body's onload event fires). Unfortunately, even if I'm correct in my flake theory, I don't know of any way (currently) to correct the run-twice failure. It seems the loader callbacks never fire when things are pulled from the cache. We'd need a way to clear the cache before this test, which I don't think we have yet. The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 88459 [details] on bug 57959. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.6 Created attachment 90733 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 90725 [details] on bug 58643. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.6 Created attachment 90759 [details]
Failure diff from eseidel-cq-sf
Given that this test fails when run twice locally, and I can't think of any good way to fix that, I think we should consider skipping it. It's possible that the flakiness could be prevented by moving the code execution to before the favicon is loaded. But it still might be flaky due to the preload scanner in that case too. Unless there is a way to clear the cache or always make the subresource loader callbacks fire, even when things are in the cache, I don't see a way to fix this test as written. The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 90874 [details] on bug 59296. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.6 Created attachment 90880 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 91082 [details] on bug 43903. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.6 Created attachment 91103 [details]
Failure diff from eseidel-cq-sf
I'll see if I can find Alice or Bradee at the conference today and we can chat about this bug. :) The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 91287 [details] on bug 59465. Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.6 Created attachment 91475 [details]
Failure diff from cr-jail-7
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 92349 [details] on bug 49902. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.6 Created attachment 92361 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 92539 [details] on bug 60338. Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.7 Created attachment 92553 [details]
Failure diff from cr-jail-8
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 92661 [details] on bug 60420. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.6 Created attachment 92689 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 92654 [details] on bug 60393. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.6 Created attachment 92726 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 92688 [details] on bug 60435. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.6 Created attachment 92738 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 92756 [details] on bug 43704. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.6 Created attachment 92760 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 92785 [details] on bug 60387. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.6 Created attachment 92791 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 91805 [details] on bug 57137. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.6 Created attachment 92844 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 92895 [details] on bug 60517. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.6 Created attachment 92904 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 92951 [details] on bug 60320. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.6 Created attachment 92977 [details]
Failure diff from eseidel-cq-sf
I've run webkit-patch find-flaky-tests and failed to find this flaking on the official bots. I will try upgrading eseidel-cq-sf to 10.6.7 and see if the problem goes away. The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 93055 [details] on bug 57127. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.7 Created attachment 93060 [details]
Failure diff from eseidel-cq-sf
Obviously that didn't work. eseidel-cq-sf is not quite a stock copy of Snow Leopard, so that may be related. However both cr-jail-8 an cr-jail-7 (stock snow leopard 2-core mac minis) have seen this failure. I do believe this is a race (due to the multi-core bots) but I don't know why it's showing up on the commit-queue when we haven't noticed it on the build.webkit.org bots yet. The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 92982 [details] on bug 60477. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.7 Created attachment 93137 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 93145 [details] on bug 60521. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.7 Created attachment 93154 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 93132 [details] on bug 60634. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.7 Created attachment 93172 [details]
Failure diff from eseidel-cq-sf
Maybe I need to shut off that bot. I think this test is likely flaky, but I don't understand why it mostly flakes on that one bot. The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 92912 [details] on bug 60508. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.7 Created attachment 93199 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 92936 [details] on bug 60542. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.7 Created attachment 93262 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 93302 [details] on bug 60519. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.7 Created attachment 93314 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 93394 [details] on bug 58999. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.7 Created attachment 93403 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 93619 [details] on bug 60874. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.7 Created attachment 93666 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 93701 [details] on bug 60926. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.7 Created attachment 93711 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 93940 [details] on bug 61080. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.7 Created attachment 93964 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 93968 [details] on bug 39852. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.7 Created attachment 93999 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 93987 [details] on bug 61003. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.7 Created attachment 94004 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 93894 [details] on bug 61034. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.7 Created attachment 94053 [details]
Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 94265 [details] on bug 42669. Bot: cr-jail-4 Port: Mac Platform: Mac OS X 10.6.7 Created attachment 94423 [details]
Failure diff from cr-jail-4
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 94506 [details] on bug 56627. Bot: cr-jail-4 Port: Mac Platform: Mac OS X 10.6.7 Created attachment 94556 [details]
Failure diff from cr-jail-4
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 94980 [details] on bug 61535. Bot: cr-jail-4 Port: Mac Platform: Mac OS X 10.6.7 Created attachment 95024 [details]
Failure diff from cr-jail-4
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 95539 [details] on bug 61835. Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.7 Created attachment 95590 [details]
Failure diff from cr-jail-8
The commit-queue just saw http/tests/misc/favicon-loads-with-icon-loading-override.html flake (Text diff mismatch) while processing attachment 96016 [details] on bug 62087. Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.7 Created attachment 96021 [details]
Failure diff from cr-jail-7
|