Bug 229399 – Composited & non-composited animation iterations are unsynced

• Component: Animations
• Status: NEW

[Matt Perry]

Created attachment 436172 [details]
Screen recording of background color iterating after transform and opacity

Visit this sandbox in Safari: https://codesandbox.io/s/waapi-webkit-iteration-timing-bug-forked-8ozd3?file=/src/index.js

This animation uses `transform`, `opacity` and `background`. This is slightly difficult to replicate but I've seen `transform` and `opacity` iterate up to two frames before or two frames after `background`. 

Expected: Values iterate start at the same time so they should *always* iterate on the same frame.

[Matt Perry]

This is also reproducible using CSS animations https://codesandbox.io/s/css-animation-webkit-iteration-timing-bug-forked-4bg8j?file=/src/styles.css:0-925

[Radar WebKit Bug Importer]

<rdar://problem/82516348>

[Antoine Quint]

This is indeed quite visible, you can see the blocks flash as yellow for one frame when at their starting position.

[Antoine Quint]

The issue with one of the rectangles using a different color while animating is fixed by bug 229398.

[Antoine Quint]

Created attachment 450766 [details]
Reduced test case

Attaching a simplified test case which clearly shows the synchronization issue with two properties being animated at once: "transform" to move the element on the y-axis (accelerated) and "margin-left" to move the element on the x-axis (non-accelerated). It's very clear that there is a frame where the "margin-left" animation is already at progress=0 while the "transform" animation is still at progress=1.

[Antoine Quint]

Adding a delay, even just 1ms, fixes the issue with the reduction, because I think we're switching from using a pre-computed absolute start time in the startAnimation() lambda in KeyframeEffect when committing the GraphicsLayerCA animation rather than using the current media time.

[Antoine Quint]

The delay thing is a complete red herring and indicative of another bug actually: we simply don't run an accelerated animation in this case! We do add a pending action to start the accelerated animation but when we try to start it, the renderer is not composited yet and we exit, the animation never starting. I expect the nature of this bug is different than the synchronization issue we're trying to sort out here, so I filed bug 236080.

[Antoine Quint]

I don't have good news to report about this bug.

While I have fixed the egregious issue with the animation for one of the sqaure not animating in the same way as the other squares with bug 229398, the cruz of the issue with the "background-color" and "transform" animations not iterating in sync is something that I don't believe we can fix with our current architecture.

When we run the "transform" animation in this case, we delegate the actual animation to Core Animation. This has some important benefits: animations run smoothly regardless of any potential delays on the main thread (think JS work or delayed rendering due to large CSS or DOM changes), are very power-efficient, and on Apple hardware with Pro Motion displays, those animations can run at a higher frame rate than those run in software.

Meanwhile, the "background-color" property animation is not delegated to Core Animation. Instead we animate this property much like we animate the bulk of CSS properties in WebKit: while the animation is "relevant" per Web Animations terminology, we schedule page rendering updates and invalidate styles for the target element such that we can animate the CSS value. The frame rate typically obtained for these types of animations is 60fps.

There are two potential synchronization issues with this approach when mixing software and "accelerated" (delegated to Core Animation) animations.

The first one, which is the crux of this bug, is that while we ensure that both kinds of animations use the same start time and are mapped correctly to the respective time origin of the worlds in which those animations live, the problem is that those animations are _not_ resolved in a coordinated manner. So while Core Animation and WebKit itself may sample animations at 60fps, within a given frame, the animations may resolve at a different iteration progress because the time within the 1/60 second that the frame may last that they are sampled is going to be different. The difference is likely a few milliseconds and for a lot of animations, this will not prove to be a problem.

However, in this case, you have those two animations running with the same timing, so when we reach the end of an iteration, one animation may be ever so-close to an iteration progress of 1, while the other animation maybe instead a hair above an iteration progress of 0.

The second potential synchronization issue is really just a different version of this first issue when, on a Pro Motion display for instance, the software and accelerated animations may run at a different frame rate, Core Animation being able to sample animations at 120fps while the software animations would still run at 60fps.

This means that while we may be off by a frame on more traditional displays, we now have the risk of being off by two frames by virtue of Core Animation sampling animations twice as fast.

There are things we could do to mitigate these issues. We could for instance try to "predict" the interval between the times where both types of animations are sampled and try to make up for this, but this wouldn't be fool-proof. I'd have to invest more time in trying to come up with a solution similar to this, but I'm not able to commit to this at the moment and cannot suggest a time when I could do this.

[Antoine Quint]

Another thing I'd like to mention about software vs. accelerated animations is that there's also some kind of a third in-between mode. There are cases where we don't know how to delegate a transform animation to Core Animation (for various reasons, some being feature incompatibilities between Core Animation and Web Animations, some being work being required on WebKit's part). When that happens, we run the "transform" animation by updating the page rendering and resolving styles at 60fps. However, where that animation will differ from animating a purely software-implemented property, such as "margin-left" for instance, is that we know how to map a CSS transform onto a Core Animation layer property. This means that while we do resolve styles, the CSS update itself may not trigger a possibly-expensive layout, but instead updates a property on a CALayer. So while performance is guaranteed to be better for animations delegated to Core Animation, running a "transform" animation in software is still going to be more efficient than animating other types of CSS properties.

[Antoine Quint]

I will attempt to fix this as part of bug 250970.