http/tests/xmlhttprequest/cross-origin-no-authorization.html is failing on Snow Leopard Release bot http://build.webkit.org/results/SnowLeopard%20Intel%20Release%20(Tests)/r52963%20(4004)/http/tests/xmlhttprequest/cross-origin-no-authorization-pretty-diff.html Seems to be a new failure (appeared after buildbot restart).
Same failure just now: http://build.webkit.org/results/SnowLeopard%20Intel%20Release%20(Tests)/r53039%20(4076)/results.html --- layout-test-results/http/tests/xmlhttprequest/cross-origin-no-authorization-expected.txt 2010-01-09 11:04:24.000000000 -0800 +++ layout-test-results/http/tests/xmlhttprequest/cross-origin-no-authorization-actual.txt 2010-01-09 11:04:24.000000000 -0800 @@ -6,7 +6,7 @@ SCRIPT SRC='...' Should succeed, since authorization is sent for cross-origin subresource loads. PASS: Loaded, user test Cross-origin XMLHttpRequest (sync), authorization will not be sent, because withCredentials is false. -PASS: 401 Authorization required +PASS: Got an exception. Error: NETWORK_ERR: XMLHttpRequest Exception 101 Cross-origin XMLHttpRequest (sync), testing authorization that's not allowed by the server (withCredentials is true, but access control headers are not set). PASS: Got an exception. Error: NETWORK_ERR: XMLHttpRequest Exception 101 Cross-origin XMLHttpRequest (sync), testing cookies.
Again tonight: http://build.webkit.org/results/SnowLeopard%20Intel%20Release%20(Tests)/r53056%20(4102)/http/tests/xmlhttprequest/cross-origin-no-authorization-diffs.txt
Another failure: http://build.webkit.org/results/SnowLeopard%20Intel%20Release%20(Tests)/r53103%20(4149)/http/tests/xmlhttprequest/cross-origin-no-authorization-diffs.txt
Different failure this time. Looks more like a timeout: http://build.webkit.org/results/SnowLeopard%20Intel%20Release%20(Tests)/r53113%20(4157)/http/tests/xmlhttprequest/cross-origin-no-authorization-diffs.txt --- layout-test-results/http/tests/xmlhttprequest/cross-origin-no-authorization-expected.txt 2010-01-11 17:37:19.000000000 -0800 +++ layout-test-results/http/tests/xmlhttprequest/cross-origin-no-authorization-actual.txt 2010-01-11 17:37:19.000000000 -0800 @@ -1,25 +1,4 @@ +CONSOLE MESSAGE: line 5: NETWORK_ERR: XMLHttpRequest Exception 101: A network error occured in synchronous requests. +FAIL: Timed out waiting for notifyDone to be called Start -Trying different ways to access a password protected resource from another origin. The UA already has login and password for this protection space. - -You should see several PASS messages followed by a DONE - -SCRIPT SRC='...' Should succeed, since authorization is sent for cross-origin subresource loads. -PASS: Loaded, user test -Cross-origin XMLHttpRequest (sync), authorization will not be sent, because withCredentials is false. -PASS: 401 Authorization required -Cross-origin XMLHttpRequest (sync), testing authorization that's not allowed by the server (withCredentials is true, but access control headers are not set). -PASS: Got an exception. Error: NETWORK_ERR: XMLHttpRequest Exception 101 -Cross-origin XMLHttpRequest (sync), testing cookies. -PASS -Cross-origin XMLHttpRequest (async), authorization will not be sent, because withCredentials is false. -PASS: Received error event. -Cross-origin XMLHttpRequest (async), testing authorization that's not allowed by the server (withCredentials is true, but access control headers are not set). -PASS: Received error event. -Cross-origin XMLHttpRequest (async), testing cookies. -PASS -Cross-origin XMLHttpRequest (sync), testing authorization with explicitly provided credentials that should be ignored. -PASS: 401 Authorization required -Cross-origin XMLHttpRequest (async), testing authorization with explicitly provided credentials that should be ignored. -PASS: Received error event. -DONE
Still failing: http://build.webkit.org/results/SnowLeopard%20Intel%20Leaks/r56894%20(5477)/http/tests/xmlhttprequest/cross-origin-no-authorization-diffs.txt --- /Volumes/Data/WebKit-BuildSlave/snowleopard-intel-leaks/build/layout-test-results/http/tests/xmlhttprequest/cross-origin-no-authorization-expected.txt 2010-03-31 22:10:31.000000000 -0700 +++ /Volumes/Data/WebKit-BuildSlave/snowleopard-intel-leaks/build/layout-test-results/http/tests/xmlhttprequest/cross-origin-no-authorization-actual.txt 2010-03-31 22:10:31.000000000 -0700 @@ -18,7 +18,7 @@ Cross-origin XMLHttpRequest (async), testing cookies. PASS Cross-origin XMLHttpRequest (sync), testing authorization with explicitly provided credentials that should be ignored. -PASS: 401 Authorization required +PASS: Got an exception. Error: NETWORK_ERR: XMLHttpRequest Exception 101 Cross-origin XMLHttpRequest (async), testing authorization with explicitly provided credentials that should be ignored. PASS: Received error event. DONE Seems it's time to skip this if we don't know why it's failing.
Created attachment 52262 [details] Patch
Comment on attachment 52262 [details] Patch I object against skipping security-related tests. Especially since the other result seems to be another kind of success at a first glance.
I'm happy to not skip it. But it's been causing random failures on the bots for 2 months now w/o comment from any of the test authors, so I am somewhat limited in my options. :)
Anyone is free to investigate this, don't blame the messenger.
What your asking is for a single test to hold the green-ness of multiple bots hostage. Bots are really only useful if we have systems in place to keep them green. If we have flaky tests which turn them intermittently red then they're only slightly more useful than non-existant bots. :) Better solutions for handling flaky tests are coming. But for now, someone trying to maintain green bots, the only tools are rollouts and skipping tests. Hopefully I will soon be able to provide you data as to when the first time we ever saw this test fail was.
It's critically important to have tests that don't let us regress security. Possibly related: bug 36843.
*** Bug 51732 has been marked as a duplicate of this bug. ***
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake while processing attachment 77807 [details] on bug 51819. Bot: cr-jail-4 Port: Mac Platform: Mac OS X 10.6.4
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 79867 [details] on bug 52977. Bot: eseidel-cq-sl Port: Mac Platform: Mac OS X 10.6.6
Created attachment 79912 [details] Failure diff from eseidel-cq-sl
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 79992 [details] on bug 46733. Bot: eseidel-cq-sl Port: Mac Platform: Mac OS X 10.6.6
Created attachment 79995 [details] Failure diff from eseidel-cq-sl
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 80613 [details] on bug 46748. Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.6
Created attachment 80712 [details] Failure diff from cr-jail-8
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 80971 [details] on bug 48719. Bot: cr-jail-4 Port: Mac Platform: Mac OS X 10.6.6
Created attachment 80986 [details] Failure diff from cr-jail-4
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 81356 [details] on bug 53849. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4
Created attachment 81367 [details] Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 81904 [details] on bug 54155. Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.6
Created attachment 81975 [details] Failure diff from cr-jail-7
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 85600 [details] on bug 56255. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4
Created attachment 85797 [details] Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 89620 [details] on bug 58259. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4
Created attachment 89739 [details] Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 89664 [details] on bug 58509. Bot: cr-jail-3 Port: Mac Platform: Mac OS X 10.6.6
Created attachment 89789 [details] Failure diff from cr-jail-3
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 90082 [details] on bug 58798. Bot: cr-jail-3 Port: Mac Platform: Mac OS X 10.6.6
Created attachment 90137 [details] Failure diff from cr-jail-3
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 90208 [details] on bug 58656. Bot: cr-jail-3 Port: Mac Platform: Mac OS X 10.6.6
Created attachment 90214 [details] Failure diff from cr-jail-3
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 90523 [details] on bug 59097. Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.6
Created attachment 90538 [details] Failure diff from cr-jail-7
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 91835 [details] on bug 59840. Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.7
Created attachment 91852 [details] Failure diff from cr-jail-8
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 91974 [details] on bug 59933. Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.7
Created attachment 92035 [details] Failure diff from cr-jail-8
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 91712 [details] on bug 58448. Bot: cr-jail-3 Port: Mac Platform: Mac OS X 10.6.7
Created attachment 92115 [details] Failure diff from cr-jail-3
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 92074 [details] on bug 60024. Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.7
Created attachment 92148 [details] Failure diff from cr-jail-7
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 92095 [details] on bug 59482. Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.7
Created attachment 92199 [details] Failure diff from cr-jail-8
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 92378 [details] on bug 60086. Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.7
Created attachment 92511 [details] Failure diff from cr-jail-7
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 92483 [details] on bug 60292. Bot: cr-jail-4 Port: Mac Platform: Mac OS X 10.6.7
Created attachment 92624 [details] Failure diff from cr-jail-4
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 92751 [details] on bug 60249. Bot: cr-jail-4 Port: Mac Platform: Mac OS X 10.6.7
Created attachment 92752 [details] Failure diff from cr-jail-4
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 92797 [details] on bug 60475. Bot: cr-jail-4 Port: Mac Platform: Mac OS X 10.6.7
Created attachment 92828 [details] Failure diff from cr-jail-4
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 92840 [details] on bug 60500. Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.6
Created attachment 92865 [details] Failure diff from eseidel-cq-sf
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 93237 [details] on bug 58999. Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.7
Created attachment 93258 [details] Failure diff from cr-jail-8
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 93707 [details] on bug 60412. Bot: cr-jail-4 Port: Mac Platform: Mac OS X 10.6.7
Created attachment 93721 [details] Failure diff from cr-jail-4
The commit-queue just saw http/tests/xmlhttprequest/cross-origin-no-authorization.html flake (Text diff mismatch) while processing attachment 94841 [details] on bug 61417. Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.7
Created attachment 94910 [details] Failure diff from cr-jail-7
Reliably fails on Win10 only: https://results.webkit.org/?suite=layout-tests&test=http/tests/xmlhttprequest/cross-origin-no-authorization.html.