animations/play-state-in-shorthand.html is a flaky failure on macOS WK2 Debug
@@ -1,4 +1,4 @@
PASS - "transform" property for "box" element at 0.5s saw something close to: 1,0,0,1,75,0
-PASS - "transform" property for "box" element at 1s saw something close to: 1,0,0,1,150,0
-PASS - "transform" property for "box" element at 2.5s saw something close to: 1,0,0,1,150,0
+FAIL - "transform" property for "box" element at 1s expected: 1,0,0,1,150,0 but saw: matrix(1, 0, 0, 1, 171.3000030517578, 0)
+FAIL - "transform" property for "box" element at 2.5s expected: 1,0,0,1,150,0 but saw: matrix(1, 0, 0, 1, 175.0500030517578, 0)
Based on the flakiness dashboard and the fact that the legacy version of this test is also marked as flaky this doesn't seem to have a recent regression.
The test itself usually take around 2 seconds, even when failing.
Test still flaky on Mac WK2 Debug
run-webkit-tests --root d240655 animations/play-state-in-shorthand.html --iterations 500 -f --debug
Test was added in https://trac.webkit.org/changeset/200043/webkit
Is flaky on most Mac builds, and recently started showing up on iOS Sim Release and Debug as well.
Updated test expectations for Mac Debug and iOS Simulator in https://trac.webkit.org/changeset/244780/webkit .
If I understand this test, it does the following (Simon, please correct me if I'm wrong):
- set a timeout for 1s in the start callback to pause the animation using the CSS `animation` property shorthand halfway through
- set various timeouts for the animated value to be tested at 500ms (before the start callback timeout has elapsed and the animation is paused), at 1000ms and 2500ms to check the animation has been paused.
So the design of this test relies exclusively on setTimeout, which is bound to be flaky, especially as both the start callback and the second value check use the same timeout.
Let's see if we can respect the design of this test while making is robust.
I think it would be best to rewrite this test for the new animation engine to check the play state of the animation rather than be looking at CSS values. I'm going to do that, and leave the legacy variety of this test alone.
Created attachment 380328 [details]
Comment on attachment 380328 [details]
Clearing flags on attachment: 380328
Committed r250781: <https://trac.webkit.org/changeset/250781>
All reviewed patches have been landed. Closing bug.