Bug 231063 - ImageBitmap should honor EXIF orientation
Summary: ImageBitmap should honor EXIF orientation
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Canvas (show other bugs)
Version: WebKit Local Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Cameron McCormack (:heycam)
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-09-30 22:28 PDT by Cameron McCormack (:heycam)
Modified: 2022-06-29 23:48 PDT (History)
13 users (show)

See Also:


Attachments
Patch (11.51 KB, patch)
2021-10-17 20:23 PDT, Cameron McCormack (:heycam)
no flags Details | Formatted Diff | Diff
Patch (12.50 KB, patch)
2021-10-17 21:34 PDT, Cameron McCormack (:heycam)
no flags Details | Formatted Diff | Diff
Patch (12.65 KB, patch)
2021-10-17 23:38 PDT, Cameron McCormack (:heycam)
mmaxfield: review+
Details | Formatted Diff | Diff
Patch for landing (20.28 KB, patch)
2021-10-18 19:48 PDT, Cameron McCormack (:heycam)
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Cameron McCormack (:heycam) 2021-09-30 22:28:34 PDT
.
Comment 1 Radar WebKit Bug Importer 2021-09-30 23:20:02 PDT
<rdar://problem/83753956>
Comment 2 Cameron McCormack (:heycam) 2021-10-12 22:12:51 PDT
Filed https://github.com/whatwg/html/issues/7210 to get clarity in the HTML spec about exactly how EXIF orientation and the image-orientation property should influence createImageBitmap.
Comment 3 Cameron McCormack (:heycam) 2021-10-17 20:23:59 PDT
Created attachment 441558 [details]
Patch
Comment 4 Cameron McCormack (:heycam) 2021-10-17 20:24:51 PDT
Submitted web-platform-tests pull request: https://github.com/web-platform-tests/wpt/pull/31287
Comment 5 EWS Watchlist 2021-10-17 20:25:05 PDT
This patch modifies the imported WPT tests. Please ensure that any changes on the tests (not coming from a WPT import) are exported to WPT. Please see https://trac.webkit.org/wiki/WPTExportProcess
Comment 6 Cameron McCormack (:heycam) 2021-10-17 21:34:30 PDT
Created attachment 441560 [details]
Patch
Comment 7 Cameron McCormack (:heycam) 2021-10-17 23:38:54 PDT
Created attachment 441561 [details]
Patch
Comment 8 Myles C. Maxfield 2021-10-18 14:02:47 PDT
Comment on attachment 441561 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=441561&action=review

> Source/WebCore/html/ImageBitmap.cpp:391
> +    auto orientation = imageForRender->orientation();
> +    if (orientation == ImageOrientation::FromImage)
> +        orientation = ImageOrientation::None;

I've had to write similar code to this elsewhere. I think we should try to push this complexity somewhere lower level. I'm not sure where, though.

> Source/WebCore/platform/graphics/ImageOrientation.h:111
> +    ImageOrientation flipY() const

Can this be named something like "ImageOrientationWithFlippedY()"? It sounds like it would be a non-const function.
Comment 9 Said Abou-Hallawa 2021-10-18 14:18:24 PDT
Comment on attachment 441561 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=441561&action=review

> Source/WebCore/html/ImageBitmap.cpp:238
> +static ImageOrientation imageOrientationForOrientation(ImageBitmapOptions::Orientation imageBitmapOrientation, ImageOrientation imageOrientation = ImageOrientation::None)

This function is confusing:
1. The name does not reveal what it converts from and what it converts to. We have ImageBitmapOptions::Orientation and ImageOrientation. So which one is "imageOrientation" and which one is "Orientation"?
2. The second name of the second parameter "imageOrientation" does not reveal what it represents. Can it be something "defaultImageOrientation"?

> Source/WebCore/html/ImageBitmap.cpp:389
> +    auto orientation = imageForRender->orientation();

imageForRender -> imageForRenderer
Comment 10 Cameron McCormack (:heycam) 2021-10-18 16:13:33 PDT
(In reply to Myles C. Maxfield from comment #8)
> I've had to write similar code to this elsewhere. I think we should try to
> push this complexity somewhere lower level. I'm not sure where, though.

It might be an artifact of Image::orientation() having a default implementation that returns FromImage.  Maybe it should be returning None.  (I'll file a followup.)
Comment 11 Cameron McCormack (:heycam) 2021-10-18 17:12:24 PDT
(In reply to Said Abou-Hallawa from comment #9)
> > Source/WebCore/html/ImageBitmap.cpp:238
> > +static ImageOrientation imageOrientationForOrientation(ImageBitmapOptions::Orientation imageBitmapOrientation, ImageOrientation imageOrientation = ImageOrientation::None)
> 
> This function is confusing:

I think you're right that the name is no longer appropriate.  I'll rename.  (But I guess I'll leave the other two similarly named functions, interpolationQualityForResizeQuality and alphaPremultiplicationForPremultiplyAlpha, since they're pure conversion functions.)
Comment 12 Cameron McCormack (:heycam) 2021-10-18 19:48:16 PDT
Created attachment 441678 [details]
Patch for landing
Comment 13 Cameron McCormack (:heycam) 2021-10-18 19:57:04 PDT
(In reply to Cameron McCormack (:heycam) from comment #10)
> (In reply to Myles C. Maxfield from comment #8)
> > I've had to write similar code to this elsewhere. I think we should try to
> > push this complexity somewhere lower level. I'm not sure where, though.
> 
> It might be an artifact of Image::orientation() having a default
> implementation that returns FromImage.  Maybe it should be returning None. 
> (I'll file a followup.)

Filed bug 231934.
Comment 14 EWS 2021-10-18 23:32:31 PDT
Committed r284436 (243198@main): <https://commits.webkit.org/243198@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 441678 [details].
Comment 15 Micah 2022-06-29 11:23:02 PDT
Anyone know when this will land in Safari?

Currently, in Safari 15, EXIF orientations 5-8 are not handled correctly. Oddly enough, 1-4 are handled just fine.

We still have to use ImageElement in Safari (instead of ImageBitmap) when rendering images to Canvas / WebGL texture to work around this bug. I'm not positive if this exact bug is tracked by this issue, but it seems to be related.
Comment 16 Cameron McCormack (:heycam) 2022-06-29 14:33:10 PDT
(In reply to Micah from comment #15)
> Anyone know when this will land in Safari?

This change was included in Safari 15.2.

> Currently, in Safari 15, EXIF orientations 5-8 are not handled correctly.
> Oddly enough, 1-4 are handled just fine.
> 
> We still have to use ImageElement in Safari (instead of ImageBitmap) when
> rendering images to Canvas / WebGL texture to work around this bug. I'm not
> positive if this exact bug is tracked by this issue, but it seems to be
> related.

Either sounds like a related issue, or the fix for this bug wasn't complete. Could you file a new bug with a description of the issue you're seeing and CC me? Thank you.
Comment 17 Micah 2022-06-29 15:04:48 PDT
(In reply to Cameron McCormack (:heycam) from comment #16)
> (In reply to Micah from comment #15)
> > Anyone know when this will land in Safari?
> 
> This change was included in Safari 15.2.
> 
> > Currently, in Safari 15, EXIF orientations 5-8 are not handled correctly.
> > Oddly enough, 1-4 are handled just fine.
> > 
> > We still have to use ImageElement in Safari (instead of ImageBitmap) when
> > rendering images to Canvas / WebGL texture to work around this bug. I'm not
> > positive if this exact bug is tracked by this issue, but it seems to be
> > related.
> 
> Either sounds like a related issue, or the fix for this bug wasn't complete.
> Could you file a new bug with a description of the issue you're seeing and
> CC me? Thank you.

Thanks Cameron! I created a new issue here:
https://bugs.webkit.org/show_bug.cgi?id=242137

Turns out I was confused about EXIF orientations 1-4 working. Looks like none of them are working (except 1 of course).