Bug 51613 – Flaky Test: http/tests/xmlhttprequest/basic-auth.html

• Component: Tools / Tests
• Status: REOPENED

[WebKit Commit Bot]

This is an automatically generated bug from the commit-queue.
http/tests/xmlhttprequest/basic-auth.html has been flaky on the commit-queue.

http/tests/xmlhttprequest/basic-auth.html was authored by [e-mail].
http://trac.webkit.org/browser/trunk/LayoutTests/http/tests/xmlhttprequest/basic-auth.html

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake while processing attachment 77437 [details] on bug 51608.
Bot: cr-jail-3  Port: Mac  Platform: Mac OS X 10.6.4

The bots will update this with information from each new failure.

If you believe this bug to be fixed or invalid, feel free to close.  The bots will re-open if the flake re-occurs.

If you would like to track this test fix with another bug, please close this bug as a duplicate.  The bots will follow the duplicate chain when making future comments.

[Eric Seidel (no email)]

I suspect this is a timeout, like seen in bug 29939.

[Alin S]

looks like a classic timeout to me… i think the bot its self might have a bug.

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake while processing attachment 77576 [details] on bug 51671.
Bot: cr-jail-3  Port: Mac  Platform: Mac OS X 10.6.4

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake while processing attachment 77924 [details] on bug 51723.
Bot: cr-jail-4  Port: Mac  Platform: Mac OS X 10.6.4

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake while processing attachment 77942 [details] on bug 51906.
Bot: cr-jail-3  Port: Mac  Platform: Mac OS X 10.6.4

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Test timed out) while processing attachment 78506 [details] on bug 52201.
Bot: cr-jail-3  Port: Mac  Platform: Mac OS X 10.6.4

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 78706 [details] on bug 52308.
Bot: eseidel-sf-cq  Port: Mac  Platform: Mac OS X 10.6.4

[WebKit Commit Bot]

Created attachment 78737 [details]
Failure diff from eseidel-sf-cq

[Eric Seidel (no email)]

Finally diff uploads are working consistently!  Hopefully this will make these failures easy to track down and fix.

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Test timed out) while processing attachment 78745 [details] on bug 51249.
Bot: cr-jail-3  Port: Mac  Platform: Mac OS X 10.6.4

[WebKit Commit Bot]

Created attachment 78797 [details]
Archive of layout-test-results from cr-jail-3

[Eric Seidel (no email)]

Comment on attachment 78797 [details]
Archive of layout-test-results from cr-jail-3

Hmm... clearly I need to work on the zip uploading a bit more.

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 79008 [details] on bug 51982.
Bot: cr-jail-4  Port: Mac  Platform: Mac OS X 10.6.4

[WebKit Commit Bot]

Created attachment 79017 [details]
Failure diff from cr-jail-4

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 79449 [details] on bug 52056.
Bot: eseidel-cq-sl  Port: Mac  Platform: Mac OS X 10.6.6

[WebKit Commit Bot]

Created attachment 79456 [details]
Failure diff from eseidel-cq-sl

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 80371 [details] on bug 53263.
Bot: eseidel-cq-sf  Port: Mac  Platform: Mac OS X 10.6.4

[WebKit Commit Bot]

Created attachment 80396 [details]
Failure diff from eseidel-cq-sf

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 83041 [details] on bug 54787.
Bot: eseidel-cq-sf  Port: Mac  Platform: Mac OS X 10.6.4

[WebKit Commit Bot]

Created attachment 83640 [details]
Failure diff from eseidel-cq-sf

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 89448 [details] on bug 58383.
Bot: cr-jail-7  Port: Mac  Platform: Mac OS X 10.6.6

[WebKit Commit Bot]

Created attachment 89550 [details]
Failure diff from cr-jail-7

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 89543 [details] on bug 58016.
Bot: cr-jail-3  Port: Mac  Platform: Mac OS X 10.6.6

[WebKit Commit Bot]

Created attachment 89720 [details]
Failure diff from cr-jail-3

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 89620 [details] on bug 58259.
Bot: cr-jail-8  Port: Mac  Platform: Mac OS X 10.6.6

[WebKit Commit Bot]

Created attachment 89746 [details]
Failure diff from cr-jail-8

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 89621 [details] on bug 53556.
Bot: eseidel-cq-sf  Port: Mac  Platform: Mac OS X 10.6.4

[WebKit Commit Bot]

Created attachment 89748 [details]
Failure diff from eseidel-cq-sf

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 89627 [details] on bug 58508.
Bot: cr-jail-3  Port: Mac  Platform: Mac OS X 10.6.6

[WebKit Commit Bot]

Created attachment 89753 [details]
Failure diff from cr-jail-3

[Adam Barth]

(In reply to comment #2)
> looks like a classic timeout to me… i think the bot its self might have a bug.

We see this flake on many different bots.  It's either a problem with the test or with WebKit.

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 89764 [details] on bug 58650.
Bot: eseidel-cq-sf  Port: Mac  Platform: Mac OS X 10.6.4

[WebKit Commit Bot]

Created attachment 89805 [details]
Failure diff from eseidel-cq-sf

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 89668 [details] on bug 58582.
Bot: cr-jail-4  Port: Mac  Platform: Mac OS X 10.6.6

[WebKit Commit Bot]

Created attachment 89905 [details]
Failure diff from cr-jail-4

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 88638 [details] on bug 58044.
Bot: eseidel-cq-sf  Port: Mac  Platform: Mac OS X 10.6.4

[WebKit Commit Bot]

Created attachment 90295 [details]
Failure diff from eseidel-cq-sf

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 90633 [details] on bug 59142.
Bot: cr-jail-4  Port: Mac  Platform: Mac OS X 10.6.6

[WebKit Commit Bot]

Created attachment 90673 [details]
Failure diff from cr-jail-4

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 90784 [details] on bug 59183.
Bot: cr-jail-8  Port: Mac  Platform: Mac OS X 10.6.6

[WebKit Commit Bot]

Created attachment 90849 [details]
Failure diff from cr-jail-8

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 91178 [details] on bug 59377.
Bot: eseidel-cq-sf  Port: Mac  Platform: Mac OS X 10.6.6

[WebKit Commit Bot]

Created attachment 91241 [details]
Failure diff from eseidel-cq-sf

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 91150 [details] on bug 59487.
Bot: cr-jail-8  Port: Mac  Platform: Mac OS X 10.6.6

[WebKit Commit Bot]

Created attachment 91243 [details]
Failure diff from cr-jail-8

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 91291 [details] on bug 59545.
Bot: cr-jail-8  Port: Mac  Platform: Mac OS X 10.6.6

[WebKit Commit Bot]

Created attachment 91314 [details]
Failure diff from cr-jail-8

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 91215 [details] on bug 59484.
Bot: eseidel-cq-sf  Port: Mac  Platform: Mac OS X 10.6.6

[WebKit Commit Bot]

Created attachment 91341 [details]
Failure diff from eseidel-cq-sf

[Eric Seidel (no email)]

A few of these show XHR throwing a NETWORK_ERR:
https://bug-51613-attachments.webkit.org/attachment.cgi?id=91241
https://bug-51613-attachments.webkit.org/attachment.cgi?id=90673
https://bug-51613-attachments.webkit.org/attachment.cgi?id=89905
https://bug-51613-attachments.webkit.org/attachment.cgi?id=89753
https://bug-51613-attachments.webkit.org/attachment.cgi?id=89720
https://bug-51613-attachments.webkit.org/attachment.cgi?id=89550
https://bug-51613-attachments.webkit.org/attachment.cgi?id=83640
https://bug-51613-attachments.webkit.org/attachment.cgi?id=80396
https://bug-51613-attachments.webkit.org/attachment.cgi?id=79017

[Eric Seidel (no email)]

Created attachment 91346 [details]
full layout-test-results.zip from processing bug 59377

I'm examining the full results.zip to look at http access logs and see if I can find anything fishy.

So far nothing about /xmlhttprequest/resources/basic-auth/basic-auth.php in the error_log.txt, but some oddities in the access_log.txt.  Still investigating.

[Eric Seidel (no email)]

OK, something *really* fishy is going on the in access_log.txt.

If you look, we see:


127.0.0.1 - - [26/Apr/2011:23:46:01 -0700] "GET /xmlhttprequest/resources/basic-auth/basic-auth.php?uid=sync6 HTTP/1.1" 401 23
127.0.0.1 - sync6 [26/Apr/2011:23:46:01 -0700] "GET /xmlhttprequest/resources/basic-auth/basic-auth.php?uid=sync6 HTTP/1.1" 200 27
127.0.0.1 - - [26/Apr/2011:23:46:01 -0700] "GET /xmlhttprequest/resources/basic-auth/basic-auth.php?uid=sync7 HTTP/1.1" 401 23

Then some 130 intervening requests:
127.0.0.1 - - [26/Apr/2011:23:44:45 -0700] "GET /multipart/resources/multipart.php?interval=0&img1=green-100x100.png&img2=abe.png&wait=100 HTTP/1.1" 200 12616
127.0.0.1 - - [26/Apr/2011:23:46:32 -0700] "GET /xmlhttprequest/binary-x-user-defined.html HTTP/1.1" 200 895

Until it seems to get along to bad-auth.html, when it seems to think that sync7 is currently the user!  Which seems to be bleeding over from basic-auth.html!

127.0.0.1 - - [26/Apr/2011:23:46:33 -0700] "POST /xmlhttprequest/resources/post-echo.cgi HTTP/1.1" 200 7
127.0.0.1 - - [26/Apr/2011:23:46:33 -0700] "GET /xmlhttprequest/failed-auth.html HTTP/1.1" 200 1991
127.0.0.1 - sync7 [26/Apr/2011:23:46:33 -0700] "GET /xmlhttprequest/resources/basic-auth/basic-auth.php?uid=login1 HTTP/1.1" 401 23
127.0.0.1 - sync7 [26/Apr/2011:23:46:33 -0700] "GET /xmlhttprequest/resources/basic-auth/basic-auth.php?uid=login1 HTTP/1.1" 401 23
127.0.0.1 - - [26/Apr/2011:23:46:33 -0700] "GET /xmlhttprequest/resources/basic-auth/basic-auth.php?uid=login2 HTTP/1.1" 401 23
127.0.0.1 - badname [26/Apr/2011:23:46:33 -0700] "GET /xmlhttprequest/resources/basic-auth/basic-auth.php?uid=login2 HTTP/1.1" 401 23


Why we're bleeding between tests?  Why the original request for sync7 seems to have returned from apache, but hit a NETWORK_ERR in our code, I don't know.

@ap, any theories?  (see the access_log.txt in the latest zip I attached above).

[Eric Seidel (no email)]

Looking earlier in the log, I see auth requests which talk to ::1 instead of 127.0.0.1.  I wonder if the ipv6 issue could be biting us here:

127.0.0.1 - - [26/Apr/2011:23:46:01 -0700] "GET /xmlhttprequest/basic-auth-nopassword.html HTTP/1.1" 200 648
::1 - - [26/Apr/2011:23:46:01 -0700] "GET /xmlhttprequest/resources/basic-auth-nouserpass/basic-auth-nouserpass.php HTTP/1.1" 401 23
::1 - user [26/Apr/2011:23:46:01 -0700] "GET /xmlhttprequest/resources/basic-auth-nouserpass/basic-auth-nouserpass.php HTTP/1.1" 200 23
127.0.0.1 - - [26/Apr/2011:23:46:01 -0700] "GET /xmlhttprequest/basic-auth-nouser.html HTTP/1.1" 200 656
::1 - - [26/Apr/2011:23:46:01 -0700] "GET /xmlhttprequest/resources/basic-auth-nouserpass/basic-auth-nouserpass.php HTTP/1.1" 401 23
::1 - "" [26/Apr/2011:23:46:01 -0700] "GET /xmlhttprequest/resources/basic-auth-nouserpass/basic-auth-nouserpass.php HTTP/1.1" 200 27

[Eric Seidel (no email)]

It seems that this test involves the network layer catching a 401 and automatically re-trying.

I think that sometimes it fails to catch the 401 and returns it up to WebCore which then causes the NETWORK_ERR and the test dies (thus possibly polluting later tests).

[Eric Seidel (no email)]

Making this test log more in bug 59651.

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 91592 [details] on bug 26539.
Bot: cr-jail-7  Port: Mac  Platform: Mac OS X 10.6.6

[WebKit Commit Bot]

Created attachment 91630 [details]
Failure diff from cr-jail-7

[Eric Seidel (no email)]

Yup.  It appears sometime the network layer just aborts for no reason:
-sync2: User: sync2, password: 123.
+sync2: req.send raised exception: Error: NETWORK_ERR: XMLHttpRequest Exception 101
+sync2: req.readyState: 4
+sync2: req.status: 0

I suspect if we had the resource load delegate callbacks enabled we would see more.  There may be other flaky tests like this which have those.  Or we could check in a version of this test which does.

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 93009 [details] on bug 60469.
Bot: cr-jail-4  Port: Mac  Platform: Mac OS X 10.6.7

[WebKit Commit Bot]

Created attachment 93128 [details]
Failure diff from cr-jail-4

[WebKit Commit Bot]

The commit-queue just saw http/tests/xmlhttprequest/basic-auth.html flake (Text diff mismatch) while processing attachment 93488 [details] on bug 60738.
Bot: eseidel-cq-sf  Port: Mac  Platform: Mac OS X 10.6.7

[WebKit Commit Bot]

Created attachment 93493 [details]
Failure diff from eseidel-cq-sf

[Alexey Proskuryakov]

This is still flaky in exactly the same way as before.

https://results.webkit.org/?suite=layout-tests&test=http%2Ftests%2Fxmlhttprequest%2Fbasic-auth.html