Bug 30029 – transition-end-event tests are all racey

• Component: New Bugs
• Status: RESOLVED FIXED

[Eric Seidel (no email)]

transitions/transition-end-event-transform.html failed on leopard bot

See https://bugs.webkit.org/show_bug.cgi?id=29967#c3

This might be related to the transitions failures we've been seeing with the Snow Leopard bots.  Bug 28461.

[Eric Seidel (no email)]

The Leopard Commit Bot just saw this again:
https://bugs.webkit.org/show_bug.cgi?id=31209#c4

[Eric Seidel (no email)]

--- /tmp/layout-test-results/transitions/transition-end-event-transform-expected.txt    2009-11-06 11:38:26.000000000 -0800
+++ /tmp/layout-test-results/transitions/transition-end-event-transform-actual.txt      2009-11-06 11:38:26.000000000 -0800
@@ -1,3 +1,4 @@
+CONSOLE MESSAGE: line 36: TypeError: Result of expression 'box' [null] is not an object.
 Initiating a 500ms transition on the transform property of box1.
 
 PASS --- [Expected] Property: -webkit-transform Target: box1 Elapsed Time: 0.5

Is the failure diff.

[Eric Seidel (no email)]

It looks like this test just uses a <script> tag in the head.  And starts the test immediately on hitting hte script tag, before the test content is parsed.  I expect that occasionally the parsing pauses due to disk access or that parsing takes longer than 0.7 seconds and thus the test fails in this way.

Seems the easy solution is to put the start in an onload handler, or to just put the script tag after the test content. :)

[Eric Seidel (no email)]

From a quick survey of the .html files in transitions/ looks like some of the transitions tests use an onload handler, but several more are victim to this type of coding error.

[Eric Seidel (no email)]

I'll eventually code up a fix, but CCing Pierre (the author of this test) so he sees it go by.  Perhaps one of the transitions folks has an opinion about how we go about fixing these types of errors for all the transitions tests.

Also, my previous statement was wrong about "parsing takes longer than 0.7 seconds".  Parsing need only take an additional 0.2 seconds between when the script was executed and when the document does its first display (and thus starts the transition).  This is why when the disk or cpu is busy (as it often is on our bots) these tests run slightly slower and get flakey.

[Eric Seidel (no email)]

Looks like this is failing occasionally on the Tiger bot too:
http://build.webkit.org/results/Tiger%20Intel%20Release/r50638%20(5963)/transitions/transition-end-event-transform-pretty-diff.html

Same reason as it did on the commit bot.

[Eric Seidel (no email)]

Failed again:
http://build.webkit.org/results/Leopard%20Intel%20Release%20(Tests)/r52629%20(8808)/results.html

I think we should consider skipping this test if it can't be easily fixed.  No sense in causing commits to fail on bots to be red.

[Csaba Osztrogonác]

Failed again:
http://build.webkit.org/results/Tiger%20Intel%20Release/r53053%20%287608%29/results.html

[Ojan Vafai]

Actually, these tests all start on window load. See the implementation of runTransitionTest. It only sets up a listener for the window's load event, then it starts the test. A number of these transition-end-event tests are flaky though. It's not clear to my why at a quick glance.

[Ojan Vafai]

OK. I'll have a patch soon. Should fix flakiness all the transition-end-event tests.

[Ojan Vafai]

Created attachment 46300 [details]
Patch

[Simon Fraser (smfr)]

Comment on attachment 46300 [details]
Patch

How long does it take to run all the tests in LayoutTests/transitions before and after this change?

[Ojan Vafai]

I did a single run with and without this patch to measure.

Before: 25.37s total testing time
After: 25.46s total testing time

This change only affects the runtime of tests that fail. Should the tests fail, some would be faster, some would be slower (mostly a bit slower).

[Ojan Vafai]

Comment on attachment 46300 [details]
Patch

I'll commit this myself.

[Ojan Vafai]

Committed r53097: <http://trac.webkit.org/changeset/53097>

[Eric Seidel (no email)]

http://build.webkit.org/results/Tiger%20Intel%20Release/r56896%20(10284)/transitions/transition-end-event-transform-diffs.txt
still looks racey on Tiger.  If you have any theories as to why, I'd love to hear them.