Bug 30029

Summary: transition-end-event tests are all racey
Product: WebKit Reporter: Eric Seidel (no email) <eric>
Component: New BugsAssignee: Nobody <webkit-unassigned>
Severity: Normal CC: cmarrin, ojan, ossy, pol, simon.fraser
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: PC   
OS: OS X 10.5   
Description Flags
Patch simon.fraser: review+, ojan: commit-queue-

Description Eric Seidel (no email) 2009-10-02 15:12:01 PDT
transitions/transition-end-event-transform.html failed on leopard bot

See https://bugs.webkit.org/show_bug.cgi?id=29967#c3

This might be related to the transitions failures we've been seeing with the Snow Leopard bots.  Bug 28461.
Comment 1 Eric Seidel (no email) 2009-11-06 11:44:25 PST
The Leopard Commit Bot just saw this again:
Comment 2 Eric Seidel (no email) 2009-11-06 11:45:52 PST
--- /tmp/layout-test-results/transitions/transition-end-event-transform-expected.txt    2009-11-06 11:38:26.000000000 -0800
+++ /tmp/layout-test-results/transitions/transition-end-event-transform-actual.txt      2009-11-06 11:38:26.000000000 -0800
@@ -1,3 +1,4 @@
+CONSOLE MESSAGE: line 36: TypeError: Result of expression 'box' [null] is not an object.
 Initiating a 500ms transition on the transform property of box1.
 PASS --- [Expected] Property: -webkit-transform Target: box1 Elapsed Time: 0.5

Is the failure diff.
Comment 3 Eric Seidel (no email) 2009-11-06 11:49:25 PST
It looks like this test just uses a <script> tag in the head.  And starts the test immediately on hitting hte script tag, before the test content is parsed.  I expect that occasionally the parsing pauses due to disk access or that parsing takes longer than 0.7 seconds and thus the test fails in this way.

Seems the easy solution is to put the start in an onload handler, or to just put the script tag after the test content. :)
Comment 4 Eric Seidel (no email) 2009-11-06 12:01:20 PST
From a quick survey of the .html files in transitions/ looks like some of the transitions tests use an onload handler, but several more are victim to this type of coding error.
Comment 5 Eric Seidel (no email) 2009-11-06 12:07:00 PST
I'll eventually code up a fix, but CCing Pierre (the author of this test) so he sees it go by.  Perhaps one of the transitions folks has an opinion about how we go about fixing these types of errors for all the transitions tests.

Also, my previous statement was wrong about "parsing takes longer than 0.7 seconds".  Parsing need only take an additional 0.2 seconds between when the script was executed and when the document does its first display (and thus starts the transition).  This is why when the disk or cpu is busy (as it often is on our bots) these tests run slightly slower and get flakey.
Comment 6 Eric Seidel (no email) 2009-11-08 22:44:05 PST
Looks like this is failing occasionally on the Tiger bot too:

Same reason as it did on the commit bot.
Comment 7 Eric Seidel (no email) 2009-12-29 12:25:40 PST
Failed again:

I think we should consider skipping this test if it can't be easily fixed.  No sense in causing commits to fail on bots to be red.
Comment 9 Ojan Vafai 2010-01-11 11:40:09 PST
Actually, these tests all start on window load. See the implementation of runTransitionTest. It only sets up a listener for the window's load event, then it starts the test. A number of these transition-end-event tests are flaky though. It's not clear to my why at a quick glance.
Comment 10 Ojan Vafai 2010-01-11 12:19:42 PST
OK. I'll have a patch soon. Should fix flakiness all the transition-end-event tests.
Comment 11 Ojan Vafai 2010-01-11 12:30:57 PST
Created attachment 46300 [details]
Comment 12 Simon Fraser (smfr) 2010-01-11 12:46:08 PST
Comment on attachment 46300 [details]

How long does it take to run all the tests in LayoutTests/transitions before and after this change?
Comment 13 Ojan Vafai 2010-01-11 13:06:47 PST
I did a single run with and without this patch to measure.

Before: 25.37s total testing time
After: 25.46s total testing time

This change only affects the runtime of tests that fail. Should the tests fail, some would be faster, some would be slower (mostly a bit slower).
Comment 14 Ojan Vafai 2010-01-11 13:52:08 PST
Comment on attachment 46300 [details]

I'll commit this myself.
Comment 15 Ojan Vafai 2010-01-11 13:54:32 PST
Committed r53097: <http://trac.webkit.org/changeset/53097>
Comment 16 Eric Seidel (no email) 2010-03-31 23:28:56 PDT
still looks racey on Tiger.  If you have any theories as to why, I'd love to hear them.