Composited elements appear pixelated when scaled up using transform

• Component: CSS
• Status: ASSIGNED ---

[Sebastian Markbåge]

If you use a CSS transition together with a CSS transform to scale up (make it larger) an object such as text or an image. Then the original (smaller) rendering is used during the transition. That causes it to appear pixelated as it scales up. When the transition finishes it settles.

It would be better if the destination size was used as the rendered base when a transform scales to a larger size.

Tested on Mac Mini (Intel).

[Simon Fraser (smfr)]

*** Bug 28214 has been marked as a duplicate of this bug. ***

[Simon Fraser (smfr)]

<rdar://problem/7225963>

[Simon Fraser (smfr)]

*** Bug 49572 has been marked as a duplicate of this bug. ***

[Simon Fraser (smfr)]

*** Bug 47991 has been marked as a duplicate of this bug. ***

[MH]

I can confirm this bug, and proposed solution, but...

...I THINK IT IS NOT THE PRIORITY!!

...when working with REAL graphic on REAL project with large SVG data, my real problem is SPEED... it is much better to see pixelated graphic for some time, than wait for the year ;-)

Especially on mobile browsers...

[Simon Fraser (smfr)]

*** Bug 56701 has been marked as a duplicate of this bug. ***

[Simon Fraser (smfr)]

<rdar://problem/6134606>

[Magnesus]

I have this problem all the time when moving, scaling or rotating images. It's a deal breaker for me. Example (when moving, a second after stopping): http://wieza.iq.pl/beforeafter.png - I'm using QtWebKit, it's there since QtWebKit 2.2.1.

[Simon Fraser (smfr)]

*** Bug 119259 has been marked as a duplicate of this bug. ***

[Simon Fraser (smfr)]

Created attachment 211852 [details]
Patch

[Build Bot]

Comment on attachment 211852 [details]
Patch

Attachment 211852 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.appspot.com/results/1824154

New failing tests:
compositing/overflow/clipping-behaviour-change-is-not-propagated-to-descendants.html
compositing/overflow/clipping-behaviour-change-is-not-propagated-to-descendants2.html

[Build Bot]

Created attachment 211856 [details]
Archive of layout-test-results from webkit-ews-15 for mac-mountainlion-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: webkit-ews-15  Port: mac-mountainlion-wk2  Platform: Mac OS X 10.8.5

[Build Bot]

Comment on attachment 211852 [details]
Patch

Attachment 211852 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.appspot.com/results/1813284

New failing tests:
compositing/overflow/clipping-behaviour-change-is-not-propagated-to-descendants.html
compositing/overflow/clipping-behaviour-change-is-not-propagated-to-descendants2.html

[Build Bot]

Created attachment 211858 [details]
Archive of layout-test-results from webkit-ews-02 for mac-mountainlion

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: webkit-ews-02  Port: mac-mountainlion  Platform: Mac OS X 10.8.5

[Dean Jackson]

Comment on attachment 211852 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=211852&action=review

lgtm

> Source/WebCore/platform/graphics/ca/GraphicsLayerCA.cpp:258
> +    TransformationMatrix::DecomposedType decomposeData;

At some point in the future, I think we should rename this TransformationMatrix::DecompositionData or something (Type doesn't make sense)

> Source/WebCore/platform/graphics/ca/GraphicsLayerCA.cpp:989
> +TransformationMatrix GraphicsLayerCA::layerTransform(const FloatPoint& position, const TransformationMatrix* customTransform) const

Is customTransform the right name? customCurrentTransform?

> Source/WebCore/platform/graphics/ca/GraphicsLayerCA.cpp:1265
> +    // After committing animations, see if we need to adjust contentScale accordingly.

Typo: contentsScale

> LayoutTests/compositing/contents-scale/animating.html:17
> +        -webkit-animation: scale 100000000s infinite linear;

We should ask for "day", "months", "years", "decades", "centuries" CSS units. Except I think they must be singular, so it would be 10000day. Weird.

> LayoutTests/platform/mac/compositing/tiling/rotated-tiled-preserve3d-clamped-expected.txt:43
> -                      (tile cache coverage 0, 0 2800 x 300)
> +                      (contentsScale 1.00)
> +                      (tile cache coverage 0, 0 2799 x 299)
>                        (tile size 512 x 512)
> -                      (top left tile 0, 0 tiles grid 6 x 1)
> +                      (top left tile 0, 0 tiles grid 5 x 1)

Why did this change?

[Simon Fraser (smfr)]

https://trac.webkit.org/r155977

[Simon Fraser (smfr)]

(In reply to comment #15)

> > LayoutTests/platform/mac/compositing/tiling/rotated-tiled-preserve3d-clamped-expected.txt:43
> > -                      (tile cache coverage 0, 0 2800 x 300)
> > +                      (contentsScale 1.00)
> > +                      (tile cache coverage 0, 0 2799 x 299)
> >                        (tile size 512 x 512)
> > -                      (top left tile 0, 0 tiles grid 6 x 1)
> > +                      (top left tile 0, 0 tiles grid 5 x 1)
> 
> Why did this change?

This was explained in the changelog. Rounding error, basically.

[WebKit Commit Bot]

Re-opened since this is blocked by bug 121515

[Alexey Proskuryakov]

Rolled out in r155994.

While most of changed results just added "(contentsScale 1.00)" (I'm wondering if that's flaky), platform/mac-wk2/tiled-drawing/fixed-background/fixed-body-background-zoomed.html changed in a bigger way.

[Simon Fraser (smfr)]

Relanded in https://trac.webkit.org/r155998

[Simon Fraser (smfr)]

Too many issues; reverted in r169229.

[Simon Fraser (smfr)]

*** Bug 135246 has been marked as a duplicate of this bug. ***

[Simon Fraser (smfr)]

*** Bug 139068 has been marked as a duplicate of this bug. ***

[Simon Fraser (smfr)]

*** Bug 180106 has been marked as a duplicate of this bug. ***

[Simon Fraser (smfr)]

*** Bug 181179 has been marked as a duplicate of this bug. ***

[Simon Fraser (smfr)]

*** Bug 192768 has been marked as a duplicate of this bug. ***

[Michael Gubik]

This bug has been open since nine years and seems quite severe to me. Are there robust workarounds?

[Takao Baba]

Any update on this? I really hope this bug is fixed soon.

[Simon Fraser (smfr)]

*** Bug 201681 has been marked as a duplicate of this bug. ***

[Sam Wemyss]

Anything more on this? It seems like there is a lot of tickets being marked as duplicates, but no updates on whether this is going to be fixed or not. :)

[Tobias Eichert]

This seems to be related to WebKit on iOS only (confirmed on 13.3.1). On MacOS, using a scaling factor greater than 1 is perfectly fine.

It's unfortunate though, as all other browsers on iOS rely on the same engine, thus having the same issue.

[Simon Fraser (smfr)]

This issue exists on both platforms.

[Michael Gubik]

This occurs on both platforms. See my attachments on bug https://bugs.webkit.org/show_bug.cgi?id=192768

[Tobias Eichert]

A few more basic observations with examples:

1) Using a scale transform with translate3D directly on the image element:
macOS: sharp image
iOS: blurry image

https://codepen.io/ironoaks/pen/ZEGjXZQ

This is related to my previous comment.

2) Using a scale transform with translate3D on a wrapper element:
macOS: blurry image
iOS: blurry image

https://codepen.io/ironoaks/pen/PoqBOzg

This is related to your example.

3) Using a scale transform with translate directly on the image element:
macOS: sharp image
iOS: sharp image

https://codepen.io/ironoaks/pen/qBdyVOp

4) Using a scale transform with translate on a wrapper element:
macOS: sharp image
iOS: sharp image

https://codepen.io/ironoaks/pen/gOpjXPq

5) Using a scale transform with translate (NOT translated3D) on a cloned image element:
macOS: sharp image
iOS: blurry image

https://3zive.csb.app/

The library I used here is a modified version of medium-zoom (https://medium-zoom.francoischalifour.com/). The original version uses a CSS transform with translate3D (resulting in an expected blurry image on iOS, s. 1)). Using translate instead of translate3D should result in sharp image rendering (s. 3) but in this case it won't.

You can edit the example here: https://codesandbox.io/s/serene-rubin-3zive

If I debug this example remotely using Safari on macOS and disable the CSS attributes "will-change" on both .medium-zoom-overlay and .medium-zoom-image--opened, as well as "opacity" on .medium-zoom--opened .medium-zoom-overlay after the transform operation has finished, then the scaled image is sharp again. Maybe you can work around this issue by telling Safari to do some kind of repaint using certain CSS attributes? I'm not sure on how to proceed from here.

[Tobias Eichert]

As for "will-change", my guess is this might also be part of the problem with regards to the stacking context:

https://developer.mozilla.org/en-US/docs/Web/CSS/will-change

I'm not very familiar with browser internals, though. Still, I'd be glad to assist in testing.

[Simon Fraser (smfr)]

*** Bug 215176 has been marked as a duplicate of this bug. ***

[Thomas Bartelmess]

Created attachment 406268 [details]
Patch

[Thomas Bartelmess]

I've created a patch (loosely based on the existing patch) that fixes the scaling issue for simple CSS scale transforms.
If I could get some review feedback if this is a sensible way to go, I am happy to add the calculations for the max scale factor during animations as well.

[Thomas Bartelmess]

Created attachment 406270 [details]
Patch

[Simon Fraser (smfr)]

My concern here is terrible performance if a page is animating scale via JS.

There has been discussion in CSSWG about this issue: https://github.com/w3c/csswg-drafts/issues/236

[Samuel]

This is such a mission critical bug for applications that rely heavily on CSS transform scale. Basically any user experience revolved around the concept of zooming might have been unacceptably broken on iOS for a long time. Personally, I find it extremely frustrating to see my users dealing with an aesthetically unpleasing rendering of HTML in 2020, specially on flagship iOS. I honestly would have expected any bug with "pixelated" on the title to be treated with a certain urgency. As a desperate developer, I would absolutely accept and cheer any provisional "-webkit-" directive just so I can have a consistent experience across devices. I am not sure how else I could help but coming here and sharing my most sincere thoughts on the issue.

[Adrian Häusler]

Created attachment 416580 [details]
Test Case in combination with ::before and css content property

[Stephen Haney]

Hey Simon and group,

Thanks for the work on this issue and I appreciate that it's more than meets the eye. I see you've tried to introduce a solution to the CSS working group by allowing the user to specify their raster scale.

I work on Modulz, a design tool that's based in the browser. Our users need to be able to zoom in on their images using nearest neighbor scaling so they can check on the pixel accuracy of their work. When an image is transformed with hardware accel, image-rendering: pixelated doesn't work in Safari (seemingly because of this issue). Right now our only option is to direct users away from Safari when they report our zoom bug to us, which is not what we want to be doing!

This issue seems to be the root of many downstream symptoms. Our case is broken `image-rendering: pixelated` — perhaps because the image-rendering logic is applied before the hardware path, which does the scaling? So the software path still sees the texture at 100% and doesn't apply the image-rendering changes? I'm speculating.

And, comically, if a stacking context container is scaled with a hardware accel transform AND an image inside it is scaled with width/height settings, the `image-rendering: pixelated` applies to the image, before the blur from this issue, yielding an extremely mauled pixelated + blurred image.

---

> My concern here is terrible performance if a page is animating scale via JS.

This is a legit concern, but I'm wondering what Chromium and Gecko are doing to avoid this — or are they just living with the performance drawbacks? From what I've read, Chromium re-renders the source image whenever it is scaled past a certain threshold away from the original scale.

Perhaps related, Chromium initially almost shipped a version of image-rendering that only worked for non-accelerated elements, but caught the error and redid the implementation to work for both unaccelerated and accelerated elements. (comment 43 of this thread on Chromium's bug tracker: https://bugs.chromium.org/p/chromium/issues/detail?id=134040#c43 )

Our concern isn't during animation. We'd be fine without resampling during a CSS transition. We just need the end result to be sharp and play nicely with the CSS `image-rendering` property.

I'm sure this is stupidly oversimplified: Is it possible to debounce the scaled redrawing, or to only do it if the scaling is above some threshold (redraw every X% away from device scale)?

This post is a giant +1 to needing some sort of fix or workaround. I hope to illustrate a real world use case and product that is blocked from supporting WK / Safari due to this issue.

Thanks again for your work on this issue. Happy to provide repro cases (although I think that is covered) or testing for patches.

[Simon Fraser (smfr)]

*** Bug 227284 has been marked as a duplicate of this bug. ***

[Simon Fraser (smfr)]

*** Bug 227284 has been marked as a duplicate of this bug. ***

[Simon Fraser (smfr)]

*** Bug 227997 has been marked as a duplicate of this bug. ***

[Florian Schulz]

Reproduction:

The rendering issues only occur when there is an element with overflowing content on the page. 

In the simple demo I created a <section> with a limited height and a toggle to switch between overflow: hidden / auto.

overflow: auto → blurred rendering
overflow: hidden → sharp

https://codepen.io/getflourish/pen/MWoJEpN

Safari 14.1.1
Safari on iOS 15

[Florian Schulz]

(In reply to mail from comment #47)

Here’s a screenshot that shows Safari Dev Tools with the compositing layers:
https://pbs.twimg.com/media/E-wuq8hWEAEDt1w?format=jpg&name=4096x4096

[Nikolas Zimmermann]

Th patch from Thomas Bartelmess is partly included in wekbit.org/b/242833 -- however behind a flag that is only enabled for the Layer-Based SVG engine at present. If we decide how to control activating the new logic, we could enable it for all content.

Missing heuristics in my patch: figure out max scale during CSS animations (already existing implementations, e.g. from Miguel). Maybe introduce a CSS prop to control the behavior for JS animations? ('accurate-rendering' / 'max-perf' ??).
Anyhow, just wanted to let you know that I've incorporated the patch into my work.

[Takao Baba]

The problem feels be getting worse on iOS 15.6.

Since this issue occurs only if the elements are composited to the layer, we avoid usage of triggering compositing e.g. canvas, filter, transform, animation. However, after upgrading to iOS 15.6 the issue also occurs even on these contents. iOS 15.6 seems to generate composited layer more aggressive.

There are some ways to ensure compositing layer such as setting "will-change: transform". Though, as far as I know there are no way to completely avoid compositing layer. 

Furthermore, the system setting "Disable hardware acceleration" has been removed on recent iOS Safari. Thus there are no workarounds now.

[Simon Fraser (smfr)]

Can you provide an example of a page that has different behavior on iOS 15.6 than earlier versions?

[Takao Baba]

I've created a reduced testcase. https://jsfiddle.net/vuncxkm0/

The page is rendered sharp on iOS 14.7.1 and 15.5 but blurred on iOS 15.6.

[Simon Fraser (smfr)]

Thank you. I filed bug 244543 for this issue. It's likely a regression from bug 241450.

[Simon Fraser (smfr)]

*** Bug 160661 has been marked as a duplicate of this bug. ***