Bug 105300 – [Skia] Animated transformations on bitmap images shake

• Status: RESOLVED
• Resolution: WONTFIX
• Component: SVG

[Matt Hinchliffe]

Animating a transform property (scale, translate etc.) of a bitmap image is shaky. The wobble as it transforms is pretty uncomfortable to view.

[Matt Hinchliffe]

Confirmed issue on iOS 6

[Philip Rogers]

Confirmed. Likely an integer clamping issue.

Downstream bug: crbug.com/166638

[Philip Rogers]

(In reply to comment #1)
> Confirmed issue on iOS 6

Can you double-check this?

[Philip Rogers]

Looks like the issue is constrained to Skia-backed platforms (not CG / iOS). ImageSkia's RESAMPLE_AWESOME clamps to integer boundaries which is great for beautiful static images but not that awesome for animations (see http://trac.webkit.org/browser/trunk/Source/WebCore/platform/graphics/skia/ImageSkia.cpp#L268). Forcing linear resampling fixes this and the bouncing goes away.

A couple ideas:
1) Add an additional resampling parameter to GraphicsContext::drawImage so we would have something like LOW (nearestneighbor), DEFAULT (linear), and STATIC_HIGH (bilinear awesome).
2) Only RESAMPLE_AWESOME for identity matrices (alternatively, only for integer scales or translates). I have a simple patch to do this but the problem will still exist for animations of width/height.

@fmalita, @schenney, opinions here?

[Matt Hinchliffe]

(In reply to comment #3)
> (In reply to comment #1)
> > Confirmed issue on iOS 6
> 
> Can you double-check this?

Yes, the issue also effects iOS 6.0.1 (checking on an iPad). I've made a better example where the issue is more apparent: http://codepen.io/joe/full/oinzb

[Stephen Chenney]

One typical way to solve this is to use fast and dirty resampling while moving and then clean it up when static. I doubt that's practical here, as we would have to detect and redraw the last frame of an animation.

Why does Skia enforce integral boundaries for bilinear resampling? That just seems pointless as the algoritihm should be able to handle any boundary position for the output. Is the issue here clamping of the input image rectangle?

[Florin Malita]

(In reply to comment #4)
> Looks like the issue is constrained to Skia-backed platforms (not CG / iOS). ImageSkia's RESAMPLE_AWESOME clamps to integer boundaries which is great for beautiful static images but not that awesome for animations (see http://trac.webkit.org/browser/trunk/Source/WebCore/platform/graphics/skia/ImageSkia.cpp#L268). Forcing linear resampling fixes this and the bouncing goes away.
> 
> A couple ideas:
> 1) Add an additional resampling parameter to GraphicsContext::drawImage so we would have something like LOW (nearestneighbor), DEFAULT (linear), and STATIC_HIGH (bilinear awesome).
> 2) Only RESAMPLE_AWESOME for identity matrices (alternatively, only for integer scales or translates). I have a simple patch to do this but the problem will still exist for animations of width/height.
> 
> @fmalita, @schenney, opinions here?

We're already limiting RESAMPLE_AWESOME to scaling/translation matrices. I guess we could limit it further to identity-only but that would diminish its awesomeness :)

I don't think making it dependent on integer scales/translations would be good though - switching between different interpolation methods during animation as we're hitting integer values is going to look much worse than what's going on now.

Maybe there's a way to pass back clamping info and adjust the sizing/position to compensate with subpixel positioning?

[Florin Malita]

(In reply to comment #6)
> One typical way to solve this is to use fast and dirty resampling while moving and then clean it up when static. I doubt that's practical here, as we would have to detect and redraw the last frame of an animation.

Wouldn't we have the same issue when animating with JS? There's no way to really know when the image turns static (if at all).

[Stephen White]

(In reply to comment #8)
> (In reply to comment #6)
> > One typical way to solve this is to use fast and dirty resampling while moving and then clean it up when static. I doubt that's practical here, as we would have to detect and redraw the last frame of an animation.
> 
> Wouldn't we have the same issue when animating with JS? There's no way to really know when the image turns static (if at all).

Regular DOM content uses the timer-based resize which animates using bilinear and switches to Lanczos when the animation is complete, as Stephen C describes.  Take a look at the ImageQualityController in RenderBoxModelObject.cpp.  The basic algorithm is: 

1)  When an object is first resized, use high-quality resampling, stash the RBMO in a hash table and set a 500ms timer.
2)  If another resize request for the same object comes in before the timer expires, it's considered to be animating, and drawn using bilinear.  The timer is kicked forward.
3)  When the timer expires without further resize requests, all hashed objects are redrawn using Lanczos.

Not sure if SVG can use this directly, but perhaps it could do something similar.