Background:
This tests that suspendAnimations and resumeAnimations work properly.
It starts an animation, suspends it on setTimeout(500), and resumes it on setTimeout(1000). To test that the suspend works, it checks the animation at .6 and .9 seconds, and then again at 1.5 seconds to test that resume worked:
// [animation-name, time, element-id, property, expected-value, tolerance]
["rotate", 0.6, "box", "webkitTransform", [1,0,0,1, 200, 0], 30],
["rotate", 0.9, "box", "webkitTransform", [1,0,0,1, 200, 0], 30],
["rotate", 1.5, "box", "webkitTransform", [1,0,0,1, 400, 0], 30],
Due to the timeouts involved, this can be flaky (we are seeing it fail about 2% of the time with values of 231 and 431, 1 pixel outside the tolerance).
Rather than tweak the tolerances more, I was thinking it might make more sense to re-write the test to more explicitly test that suspend and resume work, rather than relying on the animation values. So, something like:
At 0.6 and 0.9, rather than test for being at 200, test that the value is the same at both times (aka, it is definitely suspended), and that the value is > 0 (it did indeed start), and less < the end value (it hasn't ended).
At 1.5, rather than test for being at 400, test that the value is greater than it was at 0.6/0.9 (aka, it did resume).
Does that sound ok, or would that be losing test coverage?
This seems to fail often enough on the commit-queue to not get reported. If a test fails twice in a row, it won't be reported as flaky, as the tree will be assumed to be red instead. :)
The commit-queue just saw animations/suspend-resume-animation.html flake while processing attachment 76747[details] on bug 51173.
Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4
The commit-queue just saw animations/suspend-resume-animation.html flake while processing attachment 77006[details] on bug 51338.
Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 79863[details] on bug 52968.
Bot: eseidel-cq-sl Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 79401[details] on bug 52704.
Bot: eseidel-cq-sl Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 80460[details] on bug 53308.
Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.6
(In reply to comment #0)
> Background:
>
> This tests that suspendAnimations and resumeAnimations work properly.
>
> It starts an animation, suspends it on setTimeout(500), and resumes it on setTimeout(1000). To test that the suspend works, it checks the animation at .6 and .9 seconds, and then again at 1.5 seconds to test that resume worked:
> // [animation-name, time, element-id, property, expected-value, tolerance]
> ["rotate", 0.6, "box", "webkitTransform", [1,0,0,1, 200, 0], 30],
> ["rotate", 0.9, "box", "webkitTransform", [1,0,0,1, 200, 0], 30],
> ["rotate", 1.5, "box", "webkitTransform", [1,0,0,1, 400, 0], 30],
>
> Due to the timeouts involved, this can be flaky (we are seeing it fail about 2% of the time with values of 231 and 431, 1 pixel outside the tolerance).
>
> Rather than tweak the tolerances more, I was thinking it might make more sense to re-write the test to more explicitly test that suspend and resume work, rather than relying on the animation values. So, something like:
> At 0.6 and 0.9, rather than test for being at 200, test that the value is the same at both times (aka, it is definitely suspended), and that the value is > 0 (it did indeed start), and less < the end value (it hasn't ended).
> At 1.5, rather than test for being at 400, test that the value is greater than it was at 0.6/0.9 (aka, it did resume).
>
> Does that sound ok, or would that be losing test coverage?
I generally like the concept of your idea, but I think we really need to do some range testing. Otherwise if the animation is just not working at all it would pass. So maybe your idea of repeatability coupled with a more lenient range test would give us good coverage?
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 81298[details] on bug 53815.
Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 81928[details] on bug 54169.
Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 82117[details] on bug 54280.
Bot: cr-jail-4 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 82257[details] on bug 54351.
Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 83118[details] on bug 54842.
Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 83242[details] on bug 54909.
Bot: cr-jail-4 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 83236[details] on bug 54910.
Bot: cr-jail-3 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 83303[details] on bug 54873.
Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 83482[details] on bug 54953.
Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 83875[details] on bug 54489.
Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 84492[details] on bug 55632.
Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 85538[details] on bug 56052.
Bot: cr-jail-3 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 86081[details] on bug 56580.
Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 86919[details] on bug 56263.
Bot: cr-jail-3 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 87724[details] on bug 40239.
Bot: eseidel-cq-sf Port: Mac Platform: Mac OS X 10.6.4
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 87925[details] on bug 57666.
Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 88247[details] on bug 40261.
Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 88510[details] on bug 57877.
Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 88734[details] on bug 58098.
Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 89475[details] on bug 58073.
Bot: cr-jail-3 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 83366[details] on bug 54833.
Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 89899[details] on bug 58719.
Bot: cr-jail-3 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 90452[details] on bug 59032.
Bot: cr-jail-3 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 91529[details] on bug 58365.
Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 91529[details] on bug 58365.
Bot: cr-jail-3 Port: Mac Platform: Mac OS X 10.6.6
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 91983[details] on bug 58703.
Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.7
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 92074[details] on bug 60024.
Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.7
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 94042[details] on bug 61089.
Bot: cr-jail-3 Port: Mac Platform: Mac OS X 10.6.7
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 94069[details] on bug 59345.
Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.7
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 94583[details] on bug 60827.
Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.7
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 94617[details] on bug 61284.
Bot: cr-jail-4 Port: Mac Platform: Mac OS X 10.6.7
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 94806[details] on bug 61336.
Bot: cr-jail-8 Port: Mac Platform: Mac OS X 10.6.7
The commit-queue just saw animations/suspend-resume-animation.html flake (Text diff mismatch) while processing attachment 94319[details] on bug 31721.
Bot: cr-jail-7 Port: Mac Platform: Mac OS X 10.6.7
2010-12-16 11:04 PST, WebKit Commit Bot
2010-12-20 11:47 PST, WebKit Commit Bot
2011-01-23 14:27 PST, WebKit Commit Bot
2011-01-24 16:35 PST, WebKit Commit Bot
2011-01-31 19:26 PST, WebKit Commit Bot
2011-02-05 02:35 PST, WebKit Commit Bot
2011-02-10 04:46 PST, WebKit Commit Bot
2011-02-11 05:23 PST, WebKit Commit Bot
2011-02-13 11:31 PST, WebKit Commit Bot
2011-02-20 19:56 PST, WebKit Commit Bot
2011-02-21 21:50 PST, WebKit Commit Bot
2011-02-22 21:49 PST, WebKit Commit Bot
2011-02-22 21:51 PST, WebKit Commit Bot
2011-02-23 08:36 PST, WebKit Commit Bot
2011-02-26 06:23 PST, WebKit Commit Bot
2011-03-03 03:20 PST, WebKit Commit Bot
2011-03-14 19:15 PDT, WebKit Commit Bot
2011-03-17 16:57 PDT, WebKit Commit Bot
2011-03-25 08:08 PDT, WebKit Commit Bot
2011-03-31 09:05 PDT, WebKit Commit Bot
2011-04-01 19:34 PDT, WebKit Commit Bot
2011-04-06 00:56 PDT, WebKit Commit Bot
2011-04-06 16:02 PDT, WebKit Commit Bot
2011-04-07 22:59 PDT, WebKit Commit Bot
2011-04-14 18:03 PDT, WebKit Commit Bot
2011-04-17 16:20 PDT, WebKit Commit Bot
2011-04-18 12:14 PDT, WebKit Commit Bot
2011-04-20 18:26 PDT, WebKit Commit Bot
2011-04-28 14:11 PDT, WebKit Commit Bot
2011-04-28 14:21 PDT, WebKit Commit Bot
2011-05-03 13:05 PDT, WebKit Commit Bot
2011-05-03 15:20 PDT, WebKit Commit Bot
2011-05-19 11:46 PDT, WebKit Commit Bot
2011-05-20 06:53 PDT, WebKit Commit Bot
2011-05-24 08:11 PDT, WebKit Commit Bot
2011-05-24 11:23 PDT, WebKit Commit Bot
2011-05-25 14:45 PDT, WebKit Commit Bot
2011-05-25 21:27 PDT, WebKit Commit Bot