Bug 27684 - Composited elements appear pixelated when scaled up using transform
Summary: Composited elements appear pixelated when scaled up using transform
Status: ASSIGNED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: 528+ (Nightly build)
Hardware: Mac (Intel) OS X 10.5
: P2 Major
Assignee: Simon Fraser (smfr)
URL: http://labs.calyptus.se/Transforms/
Keywords: InRadar
: 28214 47991 49572 56701 119259 135246 139068 180106 181179 192768 201681 215176 (view as bug list)
Depends on: 121515
Blocks:
  Show dependency treegraph
 
Reported: 2009-07-25 14:56 PDT by Sebastian Markbåge
Modified: 2021-03-01 14:43 PST (History)
31 users (show)

See Also:


Attachments
Patch (110.01 KB, patch)
2013-09-16 18:39 PDT, Simon Fraser (smfr)
dino: review+
buildbot: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from webkit-ews-15 for mac-mountainlion-wk2 (478.45 KB, application/zip)
2013-09-16 19:25 PDT, Build Bot
no flags Details
Archive of layout-test-results from webkit-ews-02 for mac-mountainlion (490.03 KB, application/zip)
2013-09-16 19:48 PDT, Build Bot
no flags Details
Patch (15.23 KB, patch)
2020-08-09 07:16 PDT, Thomas Bartelmess
no flags Details | Formatted Diff | Diff
Patch (15.23 KB, patch)
2020-08-09 09:52 PDT, Thomas Bartelmess
thomas.bartelmess: review?
Details | Formatted Diff | Diff
Test Case in combination with ::before and css content property (512 bytes, text/html)
2020-12-20 06:47 PST, Adrian Häusler
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sebastian Markbåge 2009-07-25 14:56:41 PDT
If you use a CSS transition together with a CSS transform to scale up (make it larger) an object such as text or an image. Then the original (smaller) rendering is used during the transition. That causes it to appear pixelated as it scales up. When the transition finishes it settles.

It would be better if the destination size was used as the rendered base when a transform scales to a larger size.

Tested on Mac Mini (Intel).
Comment 1 Simon Fraser (smfr) 2009-08-12 08:43:50 PDT
*** Bug 28214 has been marked as a duplicate of this bug. ***
Comment 2 Simon Fraser (smfr) 2009-09-15 14:54:18 PDT
<rdar://problem/7225963>
Comment 3 Simon Fraser (smfr) 2010-11-16 16:37:26 PST
*** Bug 49572 has been marked as a duplicate of this bug. ***
Comment 4 Simon Fraser (smfr) 2010-12-06 11:16:35 PST
*** Bug 47991 has been marked as a duplicate of this bug. ***
Comment 5 MH 2011-01-25 11:34:41 PST
I can confirm this bug, and proposed solution, but...

...I THINK IT IS NOT THE PRIORITY!!

...when working with REAL graphic on REAL project with large SVG data, my real problem is SPEED... it is much better to see pixelated graphic for some time, than wait for the year ;-)

Especially on mobile browsers...
Comment 6 Simon Fraser (smfr) 2011-03-26 20:43:14 PDT
*** Bug 56701 has been marked as a duplicate of this bug. ***
Comment 7 Simon Fraser (smfr) 2011-07-15 15:57:55 PDT
<rdar://problem/6134606>
Comment 8 Magnesus 2012-02-14 02:59:56 PST
I have this problem all the time when moving, scaling or rotating images. It's a deal breaker for me. Example (when moving, a second after stopping): http://wieza.iq.pl/beforeafter.png - I'm using QtWebKit, it's there since QtWebKit 2.2.1.
Comment 9 Simon Fraser (smfr) 2013-07-30 10:27:37 PDT
*** Bug 119259 has been marked as a duplicate of this bug. ***
Comment 10 Simon Fraser (smfr) 2013-09-16 18:39:42 PDT
Created attachment 211852 [details]
Patch
Comment 11 Build Bot 2013-09-16 19:25:20 PDT
Comment on attachment 211852 [details]
Patch

Attachment 211852 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.appspot.com/results/1824154

New failing tests:
compositing/overflow/clipping-behaviour-change-is-not-propagated-to-descendants.html
compositing/overflow/clipping-behaviour-change-is-not-propagated-to-descendants2.html
Comment 12 Build Bot 2013-09-16 19:25:22 PDT
Created attachment 211856 [details]
Archive of layout-test-results from webkit-ews-15 for mac-mountainlion-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: webkit-ews-15  Port: mac-mountainlion-wk2  Platform: Mac OS X 10.8.5
Comment 13 Build Bot 2013-09-16 19:48:46 PDT
Comment on attachment 211852 [details]
Patch

Attachment 211852 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.appspot.com/results/1813284

New failing tests:
compositing/overflow/clipping-behaviour-change-is-not-propagated-to-descendants.html
compositing/overflow/clipping-behaviour-change-is-not-propagated-to-descendants2.html
Comment 14 Build Bot 2013-09-16 19:48:48 PDT
Created attachment 211858 [details]
Archive of layout-test-results from webkit-ews-02 for mac-mountainlion

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: webkit-ews-02  Port: mac-mountainlion  Platform: Mac OS X 10.8.5
Comment 15 Dean Jackson 2013-09-17 09:04:01 PDT
Comment on attachment 211852 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=211852&action=review

lgtm

> Source/WebCore/platform/graphics/ca/GraphicsLayerCA.cpp:258
> +    TransformationMatrix::DecomposedType decomposeData;

At some point in the future, I think we should rename this TransformationMatrix::DecompositionData or something (Type doesn't make sense)

> Source/WebCore/platform/graphics/ca/GraphicsLayerCA.cpp:989
> +TransformationMatrix GraphicsLayerCA::layerTransform(const FloatPoint& position, const TransformationMatrix* customTransform) const

Is customTransform the right name? customCurrentTransform?

> Source/WebCore/platform/graphics/ca/GraphicsLayerCA.cpp:1265
> +    // After committing animations, see if we need to adjust contentScale accordingly.

Typo: contentsScale

> LayoutTests/compositing/contents-scale/animating.html:17
> +        -webkit-animation: scale 100000000s infinite linear;

We should ask for "day", "months", "years", "decades", "centuries" CSS units. Except I think they must be singular, so it would be 10000day. Weird.

> LayoutTests/platform/mac/compositing/tiling/rotated-tiled-preserve3d-clamped-expected.txt:43
> -                      (tile cache coverage 0, 0 2800 x 300)
> +                      (contentsScale 1.00)
> +                      (tile cache coverage 0, 0 2799 x 299)
>                        (tile size 512 x 512)
> -                      (top left tile 0, 0 tiles grid 6 x 1)
> +                      (top left tile 0, 0 tiles grid 5 x 1)

Why did this change?
Comment 16 Simon Fraser (smfr) 2013-09-17 11:12:13 PDT
https://trac.webkit.org/r155977
Comment 17 Simon Fraser (smfr) 2013-09-17 11:17:24 PDT
(In reply to comment #15)

> > LayoutTests/platform/mac/compositing/tiling/rotated-tiled-preserve3d-clamped-expected.txt:43
> > -                      (tile cache coverage 0, 0 2800 x 300)
> > +                      (contentsScale 1.00)
> > +                      (tile cache coverage 0, 0 2799 x 299)
> >                        (tile size 512 x 512)
> > -                      (top left tile 0, 0 tiles grid 6 x 1)
> > +                      (top left tile 0, 0 tiles grid 5 x 1)
> 
> Why did this change?

This was explained in the changelog. Rounding error, basically.
Comment 18 WebKit Commit Bot 2013-09-17 13:47:11 PDT
Re-opened since this is blocked by bug 121515
Comment 19 Alexey Proskuryakov 2013-09-17 13:57:54 PDT
Rolled out in r155994.

While most of changed results just added "(contentsScale 1.00)" (I'm wondering if that's flaky), platform/mac-wk2/tiled-drawing/fixed-background/fixed-body-background-zoomed.html changed in a bigger way.
Comment 20 Simon Fraser (smfr) 2013-09-17 14:33:05 PDT
Relanded in https://trac.webkit.org/r155998
Comment 21 Simon Fraser (smfr) 2014-05-22 16:59:24 PDT
Too many issues; reverted in r169229.
Comment 22 Simon Fraser (smfr) 2014-07-24 12:45:51 PDT
*** Bug 135246 has been marked as a duplicate of this bug. ***
Comment 23 Simon Fraser (smfr) 2014-12-01 11:10:00 PST
*** Bug 139068 has been marked as a duplicate of this bug. ***
Comment 24 Simon Fraser (smfr) 2017-11-28 13:59:44 PST
*** Bug 180106 has been marked as a duplicate of this bug. ***
Comment 25 Simon Fraser (smfr) 2018-01-01 18:21:35 PST
*** Bug 181179 has been marked as a duplicate of this bug. ***
Comment 26 Simon Fraser (smfr) 2018-12-17 14:39:21 PST
*** Bug 192768 has been marked as a duplicate of this bug. ***
Comment 27 Michael Gubik 2018-12-17 23:51:03 PST
This bug has been open since nine years and seems quite severe to me. Are there robust workarounds?
Comment 28 Takao Baba 2019-03-22 00:37:18 PDT
Any update on this? I really hope this bug is fixed soon.
Comment 29 Simon Fraser (smfr) 2019-09-11 15:29:11 PDT
*** Bug 201681 has been marked as a duplicate of this bug. ***
Comment 30 Sam Wemyss 2019-09-11 19:06:48 PDT
Anything more on this? It seems like there is a lot of tickets being marked as duplicates, but no updates on whether this is going to be fixed or not. :)
Comment 31 Tobias Eichert 2020-03-19 10:57:08 PDT
This seems to be related to WebKit on iOS only (confirmed on 13.3.1). On MacOS, using a scaling factor greater than 1 is perfectly fine.

It's unfortunate though, as all other browsers on iOS rely on the same engine, thus having the same issue.
Comment 32 Simon Fraser (smfr) 2020-03-19 20:01:45 PDT
This issue exists on both platforms.
Comment 33 Michael Gubik 2020-03-19 20:34:28 PDT
This occurs on both platforms. See my attachments on bug https://bugs.webkit.org/show_bug.cgi?id=192768
Comment 34 Tobias Eichert 2020-03-20 04:23:18 PDT
A few more basic observations with examples:

1) Using a scale transform with translate3D directly on the image element:
macOS: sharp image
iOS: blurry image

https://codepen.io/ironoaks/pen/ZEGjXZQ

This is related to my previous comment.

2) Using a scale transform with translate3D on a wrapper element:
macOS: blurry image
iOS: blurry image

https://codepen.io/ironoaks/pen/PoqBOzg

This is related to your example.

3) Using a scale transform with translate directly on the image element:
macOS: sharp image
iOS: sharp image

https://codepen.io/ironoaks/pen/qBdyVOp

4) Using a scale transform with translate on a wrapper element:
macOS: sharp image
iOS: sharp image

https://codepen.io/ironoaks/pen/gOpjXPq

5) Using a scale transform with translate (NOT translated3D) on a cloned image element:
macOS: sharp image
iOS: blurry image

https://3zive.csb.app/

The library I used here is a modified version of medium-zoom (https://medium-zoom.francoischalifour.com/). The original version uses a CSS transform with translate3D (resulting in an expected blurry image on iOS, s. 1)). Using translate instead of translate3D should result in sharp image rendering (s. 3) but in this case it won't.

You can edit the example here: https://codesandbox.io/s/serene-rubin-3zive

If I debug this example remotely using Safari on macOS and disable the CSS attributes "will-change" on both .medium-zoom-overlay and .medium-zoom-image--opened, as well as "opacity" on .medium-zoom--opened .medium-zoom-overlay after the transform operation has finished, then the scaled image is sharp again. Maybe you can work around this issue by telling Safari to do some kind of repaint using certain CSS attributes? I'm not sure on how to proceed from here.
Comment 35 Tobias Eichert 2020-03-20 09:20:02 PDT
As for "will-change", my guess is this might also be part of the problem with regards to the stacking context:

https://developer.mozilla.org/en-US/docs/Web/CSS/will-change

I'm not very familiar with browser internals, though. Still, I'd be glad to assist in testing.
Comment 36 Simon Fraser (smfr) 2020-08-06 12:04:26 PDT
*** Bug 215176 has been marked as a duplicate of this bug. ***
Comment 37 Thomas Bartelmess 2020-08-09 07:16:38 PDT
Created attachment 406268 [details]
Patch
Comment 38 Thomas Bartelmess 2020-08-09 07:18:41 PDT
I've created a patch (loosely based on the existing patch) that fixes the scaling issue for simple CSS scale transforms.
If I could get some review feedback if this is a sensible way to go, I am happy to add the calculations for the max scale factor during animations as well.
Comment 39 Thomas Bartelmess 2020-08-09 09:52:31 PDT
Created attachment 406270 [details]
Patch
Comment 40 Simon Fraser (smfr) 2020-08-10 10:22:19 PDT
My concern here is terrible performance if a page is animating scale via JS.

There has been discussion in CSSWG about this issue: https://github.com/w3c/csswg-drafts/issues/236
Comment 41 Samuel 2020-11-17 17:43:04 PST
This is such a mission critical bug for applications that rely heavily on CSS transform scale. Basically any user experience revolved around the concept of zooming might have been unacceptably broken on iOS for a long time. Personally, I find it extremely frustrating to see my users dealing with an aesthetically unpleasing rendering of HTML in 2020, specially on flagship iOS. I honestly would have expected any bug with "pixelated" on the title to be treated with a certain urgency. As a desperate developer, I would absolutely accept and cheer any provisional "-webkit-" directive just so I can have a consistent experience across devices. I am not sure how else I could help but coming here and sharing my most sincere thoughts on the issue.
Comment 42 Adrian Häusler 2020-12-20 06:47:05 PST
Created attachment 416580 [details]
Test Case in combination with ::before and css content property
Comment 43 Stephen Haney 2021-03-01 14:43:18 PST
Hey Simon and group,

Thanks for the work on this issue and I appreciate that it's more than meets the eye. I see you've tried to introduce a solution to the CSS working group by allowing the user to specify their raster scale.

I work on Modulz, a design tool that's based in the browser. Our users need to be able to zoom in on their images using nearest neighbor scaling so they can check on the pixel accuracy of their work. When an image is transformed with hardware accel, image-rendering: pixelated doesn't work in Safari (seemingly because of this issue). Right now our only option is to direct users away from Safari when they report our zoom bug to us, which is not what we want to be doing!

This issue seems to be the root of many downstream symptoms. Our case is broken `image-rendering: pixelated` — perhaps because the image-rendering logic is applied before the hardware path, which does the scaling? So the software path still sees the texture at 100% and doesn't apply the image-rendering changes? I'm speculating.

And, comically, if a stacking context container is scaled with a hardware accel transform AND an image inside it is scaled with width/height settings, the `image-rendering: pixelated` applies to the image, before the blur from this issue, yielding an extremely mauled pixelated + blurred image.

---

> My concern here is terrible performance if a page is animating scale via JS.

This is a legit concern, but I'm wondering what Chromium and Gecko are doing to avoid this — or are they just living with the performance drawbacks? From what I've read, Chromium re-renders the source image whenever it is scaled past a certain threshold away from the original scale.

Perhaps related, Chromium initially almost shipped a version of image-rendering that only worked for non-accelerated elements, but caught the error and redid the implementation to work for both unaccelerated and accelerated elements. (comment 43 of this thread on Chromium's bug tracker: https://bugs.chromium.org/p/chromium/issues/detail?id=134040#c43 )

Our concern isn't during animation. We'd be fine without resampling during a CSS transition. We just need the end result to be sharp and play nicely with the CSS `image-rendering` property.

I'm sure this is stupidly oversimplified: Is it possible to debounce the scaled redrawing, or to only do it if the scaling is above some threshold (redraw every X% away from device scale)?

This post is a giant +1 to needing some sort of fix or workaround. I hope to illustrate a real world use case and product that is blocked from supporting WK / Safari due to this issue.

Thanks again for your work on this issue. Happy to provide repro cases (although I think that is covered) or testing for patches.