Bug 51238 – Flaky Test: http/tests/misc/redirect.php

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[WebKit Commit Bot]

This is an automatically generated bug from the commit-queue.
http/tests/misc/redirect.php has been flaky on the commit-queue.

http/tests/misc/redirect.php was authored by [e-mail].
http://trac.webkit.org/browser/trunk/LayoutTests/http/tests/misc/redirect.php

The commit-queue just saw http/tests/misc/redirect.php flake while processing attachment 76763 [details] on bug 51184.
Bot: abarth-cq-sl  Port: Mac  Platform: Mac OS X 10.6.5

The bots will update this with information from each new failure.

If you would like to track this test fix with another bug, please close this bug as a duplicate.

[WebKit Commit Bot]

Created attachment 76856 [details]
Failure diff from abarth-cq-sl

[WebKit Commit Bot]

The commit-queue just saw http/tests/misc/redirect.php flake while processing attachment 76289 [details] on bug 50419.
Bot: cr-jail-4  Port: Mac  Platform: Mac OS X 10.6.4

[WebKit Commit Bot]

Created attachment 76857 [details]
Failure diff from cr-jail-4

[Alexey Proskuryakov]

-PASS: User-Agent header contains the string 'WebKit'.
+FAIL: User-Agent header does not contain the string 'WebKit', value is 'Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 6.0)'

I can't begin to imagine how WebKit could send such a user agent. Is there someone accessing the commit bot's httpd from outside? That should also be impossible, as (old-)run-webkit-tests only binds Apache to loopback interface by default!

[Alexey Proskuryakov]

Per <http://www.botsvsbrowsers.com/details/108410/index.html>, this UA belongs to a known bot, so the commit queue server is accepting external connections on port 8000, and somehow binds the test Apache instance to all interfaces (how?!)

If accurate, this sounds really bad to me.

[Eric Seidel (no email)]

These are two separate failures.  The first is from Adam's machine, which is in our livingroom (and is no longer being used as a regular commit queue).

The second is from the cr-jail.  I don't know much about the network topology there.

I believe that the http server we use for layout tests does open port 8000 on a non-loopback interface.

When you say the user agent is from a "known bot" I'm not sure that's correct.  The page you cite "http://www.botsvsbrowsers.com/details/108410/index.html" talks about a longer user agent, including a <script> tag.

[Alexey Proskuryakov]

> I believe that the http server we use for layout tests does open port 8000 on a non-loopback interface.

What server is that? As mentioned before, run-webkit-tests doesn't bind to any other interfaces, and for a very good reason. Code that we have in tests certainly isn't secure, and an attacker can most likely gain access to the machine they run on.

> I don't know much about the network topology there.

Who is the person who controls the commit queue then?

> a longer user agent, including a <script> tag

Indeed. Still, this doesn't look like something sent by DumpRenderTree, and must be coming from outside.

[Eric Seidel (no email)]

(In reply to comment #7)
> > I believe that the http server we use for layout tests does open port 8000 on a non-loopback interface.
> 
> What server is that? As mentioned before, run-webkit-tests doesn't bind to any other interfaces, and for a very good reason. Code that we have in tests certainly isn't secure, and an attacker can most likely gain access to the machine they run on.

Interesting.  I just tested locally to confirm.  At least run-webkit-httpd does not bind to anything but loopback, nice.

> > I don't know much about the network topology there.
> 
> Who is the person who controls the commit queue then?

I do.  But I don't control the cr-jail network.

> > a longer user agent, including a <script> tag
> 
> Indeed. Still, this doesn't look like something sent by DumpRenderTree, and must be coming from outside.

Yes, but if my if my testing is to be believed, then it must be coming from the same machine?  I do control the machine. :)

[Eric Seidel (no email)]

I think this request is coming from:
http://trac.webkit.org/browser/trunk/LayoutTests/http/tests/inspector/extensions-headers.html

[Eric Seidel (no email)]

caseq updated that test 12 hours ago after the commit-queue reported it flaky:
http://trac.webkit.org/changeset/74239

I suspect it's also bleeding into other tests.

In general, the inspector tests make up more than 1/3rd of all flaky tests at the moment. :(

[Alexey Proskuryakov]

That's a relief, thanks for finding where the request comes from!

[Alexey Proskuryakov]

Ideally, the inspector test should have used a custom header to make it clear where it comes from, to be less confusing in a failure case like this.

[Andrey Kosyakov]

(In reply to comment #12)
> Ideally, the inspector test should have used a custom header to make it clear where it comes from, to be less confusing in a failure case like this.

It actually uses a custom header _and_ a user agent -- the latter was added to see if it's not overwritten anywhere in the network stack (this happened to be the case for chromium). In fact, if not for the failure of this test, we wouldn't have noticed that we don't clean up user-supplied extra headers when inspector front-end is closed.

So the bottom line -- this wasn't a flakiness at all, just pure failure as a result of bug in inspector (fixed in r74256) and a lack of cleanup in inspector/extensions-headers.html (fixed in r74237), so I'm closing this. Sorry for the mess -- I realized this failure will cause a lot of confusion when I saw it, but didn't see this bug before Eric added me.

As a note -- this test also failed on the "core" bots, which, I guess, could have been used to tell between flakiness and failure. I've also had one of my tests detected as flaky when it was actually failing for a good reason. Can this bot be fixed to consider tests as flaky only in case these were not failing during build of a base revision on core bots? I guess this would improve s/n ratio a bit...

[Eric Seidel (no email)]

OK.  So the assumption is this report was a symptom of bug 51272 then.

[Andrey Kosyakov]

(In reply to comment #14)
> OK.  So the assumption is this report was a symptom of bug 51272 then.

Yes, thanks for the pointer!

[Andrey Kosyakov]

(In reply to comment #14)
> OK.  So the assumption is this report was a symptom of bug 51272 then.

On a second thought, this one is different: there is no other failing test, the original test that caused failure (extensions-headers.html) was passing ok, just causing side-effects on other tests by leaving custom header overrides (a few revisions later, there was a failure in extensions-headers.html, but it's unrelated to this problem).