Bug 245858 - HDR data is clipped in AVIF images
Summary: HDR data is clipped in AVIF images
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Images (show other bugs)
Version: Safari 16
Hardware: iPhone / iPad iOS 16
: P2 Normal
Assignee: Said Abou-Hallawa
URL:
Keywords: InRadar
Depends on: 252129 252182 252382
Blocks:
  Show dependency treegraph
 
Reported: 2022-09-29 18:50 PDT by Greg
Modified: 2023-12-04 09:17 PST (History)
8 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Greg 2022-09-29 18:50:59 PDT
The AVIF file format not only supports HDR (brighter than white) content, it is a critical way to share such content on the web as legacy formats such as JPG are not suitable.

iOS16 does not support HDR content in AVIF images. The HDR values are clipped (and transparency is also ignored).

Here's a Netflix AVIF which does render properly HDR in Chrome on MacOS, but not under iOS 16: https://raw.githubusercontent.com/AOMediaCodec/av1-avif/master/testFiles/Netflix
The image should be viewed on an an HDR display, such as the XDR display on the 2021 M1 MacBook Pro.

Having support would be enormously beneficial, as there are now many millions iPhones and iPads which have HDR displays - but there is no valid way to view HDR images on a web browser on any of these devices.
Comment 1 Karl Dubost 2022-09-29 20:03:24 PDT
Hi Greg, 

Thanks for the report.
Does it break Netflix? Or Netflix sends an alternate image for Safari?
Comment 2 Greg 2022-09-29 20:36:52 PDT
Hi Karl-

I am not sure which assets they are using on their domain. This is just a handy reference image which is valid but will not render with WebKit (Safari and Chrome on iOS will not render this properly, but Chrome desktop will). My own personal use as as a content creator (photographer). I’ve generated and tested other HDR AVIF images which fail in a similar manner.

I have yet to find any iOS app to render an HDR image of any kind correctly. There is an odd exception to this. The camera roll will show proper HDR thumbnails (but not the full image, only its thumbnail), and only if the active image is not an HDR or the next image to the right of an HDR. So in this narrow context, I have seen HDR AVIF and EXR thumbnails properly show as bright images in the filmstrip.  But never in any mobile browser. 

-Greg
Comment 3 Said Abou-Hallawa 2022-09-30 19:01:05 PDT
The link https://raw.githubusercontent.com/AOMediaCodec/av1-avif/master/testFiles/Netflix is broken. Can you attach an HDR AVIF image or put another working link?
Comment 4 Greg 2022-09-30 19:46:45 PDT
Here's an alternative HDR AVIF: https://drive.google.com/file/d/1S1zHnz6oNKn_DEPaDQ1_VjGNh8p6Y-z-/view?usp=sharing
Comment 5 Radar WebKit Bug Importer 2022-10-06 18:51:18 PDT
<rdar://problem/100879401>
Comment 6 Myles C. Maxfield 2022-10-14 14:12:06 PDT
Is there something specific about AVIF that this bug is about, or is this bug about clipping on _any_ HDR image?
Comment 7 Greg 2022-10-14 15:00:25 PDT
Probably HDR in general. I have not tested the experimental JXL build (nor looked for other possible formats which might be valid for HDR and supported by the browser).
Comment 8 Greg 2022-10-14 15:02:31 PDT
ignore the JXL comment, was thinking of FireFox. I don't know if there's any public test for JXL in WebKit. Regardless, I have not tested other HDR formats beyond AVIF in Safari.
Comment 9 Greg 2022-10-21 08:21:46 PDT
Here's a site with several HDR AVIF images for testing (renders properly in Chrome/Brave):
https://gregbenzphotography.com/hdr/
Comment 10 Christopher Cameron 2022-11-22 04:16:31 PST
The following test program shows the core issue:
https://github.com/ccameron-chromium/test-programs/blob/main/hlg-tone-map-simple.mm

That test program has 3 gradients:
- a CAMetalLayer that applies the HLG toLinear function
- a CAMetalLayer that applies to HLG toLinear function and sets its CAEDRMetalLayer to HLG
- an AVSampleBufferDisplayLayer that is HLG

The bottom two gradients looks the same, but the top one ramps up too quickly and then saturates.

In the top one, we're just applying the HLG inverse OETF directly to the pixel values, giving us values in the range [0, 12]. The HDR-to-SDR ratio of the display is not taken into account, so it will clip wherever we hit the EDR limit (pixel value 0.5 on SDR screens). This is what it appears is happening with HLG images in STP.

In the middle one, we're applying the HLG inverse OETF, and then setting CAEDRMetadata -- this uses the full display range.

In the bottom one, we tell AVSampleBufferDisplayLayer "do your thing", and it does the same thing as the middle layer.

The difficulty here is that when CoreGraphics is drawing HLG images, it needs to "bake in" the tone mapping that would ordinarily be applied by CoreAnimation. This also means that if the viewing conditions change, the image must be re-drawn using CG.

Another test page for this is:
https://ccameron-chromium.github.io/hdr-jpeg/index.html
Comment 11 Xin Liu 2022-12-21 06:08:14 PST
Here is another AVIF HDR test page:

https://people.csail.mit.edu/ericchan/hdr/hdr.php

It seems to be working on my iPhone 13 Pro Max with iOS 16 but not macOS Ventura Safari 16.
Comment 12 Greg 2022-12-21 06:22:49 PST
FireFox on iOS will render the images at https://people.csail.mit.edu/ericchan/hdr/hdr.php without being extremely dark, but they are still rendered incorrectly.

Open that URL in Chrome on MacOS and take a look at at clouds in the B&W mountain shot or the sunlit peaks in the shot just below it. The HDR detail isn't properly managed in any browser I've tested on iOS (including Chrome iOS).
Comment 13 Greg 2022-12-21 06:27:45 PST
Here's a test page with images encoded with ACR v15.1: https://gregbenzphotography.com/hdr-gallery/

If viewed on an HDR monitor, the page will render HDR AVIF images (if viewed on an SDR monitor, it will serve SDR JPG, but you can click the green button towards the bottom left to force it to render the HDR AVIF). These images include new metadata that help further improve rendering of HDR images. When viewed with the latest Chrome Dev or Chrome Canary, they show a notable improvement even over rendering in browser like Chrome that already supports HDR. This is a separate issue tracked in https://bugzilla.mozilla.org/show_bug.cgi?id=1805436.
Comment 14 Said Abou-Hallawa 2023-02-14 12:27:03 PST
Pull request: https://github.com/WebKit/WebKit/pull/10105
Comment 15 EWS 2023-02-18 09:02:13 PST
Committed 260512@main (2ba4ecb0408c): <https://commits.webkit.org/260512@main>

Reviewed commits have been landed. Closing PR #10105 and removing active labels.
Comment 16 Rado 2023-12-04 08:24:19 PST
Safari still doesn't display HDR AVIF, while Chrome supports it properly. This issue seems to be closed, should I open a new one? Thanks.
Comment 17 Greg 2023-12-04 09:17:57 PST
Agreed, this is an important issue to address. Lack of support in WebKit means no ability to properly see HDR photos when browsing with iPhone / iPad (these same devices look great with the HDR image support in the iOS 17 Photos app). And while Apple computer owners can work around this by using several alternatives to Safari, it is also not possible to easily see HDR photos in an Apple retail store either (which makes it hard to appreciate the benefit of XDR displays).


Issues created for missing HDR support in WebKit:
HDR AVIF: https://bugs.webkit.org/show_bug.cgi?id=265796
HDDR JPG gain maps: https://bugs.webkit.org/show_bug.cgi?id=265797