Bug 94532 - Layout test fast/forms/formmethod-attribute-button-html.html failing intermittently with missing expectations
Summary: Layout test fast/forms/formmethod-attribute-button-html.html failing intermit...
Status: RESOLVED DUPLICATE of bug 72039
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-08-20 14:56 PDT by Kenneth Russell
Modified: 2012-08-21 14:38 PDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kenneth Russell 2012-08-20 14:56:53 PDT
The layout test fast/forms/formmethod-attribute-button-html.html is failing intermittently on Linux. The flakiness dashboard thinks that it's missing results:

http://build.chromium.org/p/chromium.webkit/builders/Webkit%20Linux/builds/29834
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=fast%2Fforms%2Fformmethod-attribute-button-html.html

But what really appears to be happening is that the test is outputting the layer tree as text instead of the test's results. This is a pretty weird kind of failure. It looks similar to ones we've seen before where DRT might be picking up the results from the previous test run on that machine.
Comment 1 Kenneth Russell 2012-08-20 15:10:40 PDT
Committed r126077: <http://trac.webkit.org/changeset/126077>
Comment 2 Dirk Pranke 2012-08-20 15:26:00 PDT
Are you possibly seeing https://bugs.webkit.org/show_bug.cgi?id=63104 ?
Comment 3 Kenneth Russell 2012-08-20 15:46:22 PDT
Not sure. From reading https://bugs.webkit.org/show_bug.cgi?id=63104 it sounds like it happens reliably when the test fails before calling testRunner.dumpAsText(). This failure is intermittent. fast/forms/formmethod-attribute-button-html.html uses testRunner.waitUntilDone() / notifyDone(), though, so if the failure occurred between them, and the test didn't call notifyDone(), would that have the same effect?
Comment 4 Dirk Pranke 2012-08-20 15:54:12 PDT
dumpAsText is called from js-test-pre; if that script isn't loaded, you'll see the problem you're seeing. Which is the same general class of symptoms described in that bug (at least as far as I'm concerned).

These failures always seem to be flaky.
Comment 5 Kenneth Russell 2012-08-20 16:13:47 PDT
Sounds like this is a duplicate of the other bug in that case.

*** This bug has been marked as a duplicate of bug 63104 ***
Comment 6 Ojan Vafai 2012-08-20 17:00:22 PDT
I think this is more accurately a dupe of bug 72039. There is some flakiness issue where dumpAsText isn't getting executed. The fact that we report it as MISSING instead of some other failure is incidental. The test JS is deterministic and should *always* be executing dumpAsText. Seems there is likely a DRT/testRunner bug here.

*** This bug has been marked as a duplicate of bug 72039 ***
Comment 7 Dirk Pranke 2012-08-20 17:09:12 PDT
how is bug 72039 different from bug 63104 ?
Comment 8 Ojan Vafai 2012-08-21 12:19:39 PDT
(In reply to comment #7)
> how is bug 72039 different from bug 63104 ?

Bug 63104 could result from someone writing a bad test, e.g. a test that doesn't call dumpAsText when there's a JavaScript error. The fix for this is giving the clearest error possible.

Bug 72039 is about some issue where dumpAsText should 100% be getting called in the test and we don't dumpAsText for some reason. The fix for this is figuring out why the dumpAsText isn't registering correctly, e.g. maybe DRT isn't reseting correctly in some cases and it's getting the setting from a previous (or following!!) test. The symptoms are similar, but they are totally different bugs as I see it.

Maybe you feel they should both be about the flakiness issue, in which case we should just close bug 63104 as wontfix.
Comment 9 Dirk Pranke 2012-08-21 14:38:32 PDT
(In reply to comment #8)
> (In reply to comment #7)
> > how is bug 72039 different from bug 63104 ?
> 
> Bug 63104 could result from someone writing a bad test, e.g. a test that doesn't call dumpAsText when there's a JavaScript error. The fix for this is giving the clearest error possible.
> 
> Bug 72039 is about some issue where dumpAsText should 100% be getting called in the test and we don't dumpAsText for some reason. The fix for this is figuring out why the dumpAsText isn't registering correctly, e.g. maybe DRT isn't reseting correctly in some cases and it's getting the setting from a previous (or following!!) test. The symptoms are similar, but they are totally different bugs as I see it.
> 
> Maybe you feel they should both be about the flakiness issue, in which case we should just close bug 63104 as wontfix.

Good points. You're right, it's a dup of 72039.