Bug 107328

Summary: [chromium] DumpRenderTree is getting the contents of a previous test
Product: WebKit Reporter: Hin-Chung Lam <hclam>
Component: Tools / TestsAssignee: Nate Chapin <japhet>
Status: RESOLVED WONTFIX    
Severity: Normal CC: abarth, dpranke, japhet, schenney, tony
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   

Description Hin-Chung Lam 2013-01-18 14:38:35 PST
This test is reported to be MISSING:

http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Fcache%2Fcancel-multiple-post-xhrs.html

But however the actual results comes from a previous test run.
Comment 1 Hin-Chung Lam 2013-01-18 14:44:46 PST
Committed r140210: <http://trac.webkit.org/changeset/140210>
Comment 2 Dirk Pranke 2013-01-29 13:12:01 PST
So, what seems to be happening here is that http/tests/cache/cancel-during-revalidation-succeeded.html is somehow messing up state inside DRT so that when we run the next test (http/tests/cache/cancel-multiple-post-xhrs.html), the dumpAsText() call is not having any effect, and we're incorrectly dumping out a render tree and generating a pixel result (and since there's no -expected.png, we report this as MISSING). If you look at the actual results being generated on the flakiness dashboard, you can see the incorrect render tree.

This is kinda interesting, since it's at least the first case I've seen which deterministically (reliably) is causing dumpAsText() to have no effect and we can't blame this on a script not getting loaded.

As an aside, the second test (cancel-multiple-post-xhrs) is also printing out output  inside WebProxyTestBase::willSendRequest) *before* we print the test headers in TestEventPrinter::handleTextHeader(). This is probably bad, not the least of which is because the NRWT driver.py code is handling this only by accident.

So, there's a couple of different bugs being revealed here.
Comment 3 Dirk Pranke 2013-01-29 13:16:21 PST
Note that the http/tests/cache/cancel-during-revalidation-succeeded.html appears to be a known problem (at least on linux) as noted from bug 96950. 

Also, the "actual results come from a previous test run" part refers the output from cancel-during-revalidation-succeeded.html showing up as the -actual for cancel-multiple-post-xhrs.html .
Comment 4 Dirk Pranke 2013-01-29 14:19:50 PST
Punting this to Nate ... it appears that both NRWT and DRT are working correctly, and when we attempt to load the second URL, we're getting the contents of the first URL (in some form) back. 

I am able to reproduce this consistently on my mac sync'ed to r141137 in a release build. It does not appear to reproduce in a debug build at the same revision (at least, not easily).

run-webkit-tests --chromium --no-new-test-results --clobber-old-results http/tests/cache/cancel-during-revalidation-succeeded.html http/tests/cache/cancel-multiple-post-xhrs.html
Comment 5 Dirk Pranke 2013-01-29 14:21:03 PST
Also, editing the subject line to reflect my current understanding.

Was: "Layout test runner is confused with output from different tests"
Now: "[chromium] DumpRenderTree is getting the contents of a previous test"

I have no idea yet if this reproduces on other ports.
Comment 6 Dirk Pranke 2013-01-29 14:43:15 PST
It does not appear to reproduce on the apple mac port at the same revision.