Bug 245858 – HDR data is clipped in AVIF images

• Status: RESOLVED
• Resolution: FIXED
• Component: Images

[Greg]

The AVIF file format not only supports HDR (brighter than white) content, it is a critical way to share such content on the web as legacy formats such as JPG are not suitable.

iOS16 does not support HDR content in AVIF images. The HDR values are clipped (and transparency is also ignored).

Here's a Netflix AVIF which does render properly HDR in Chrome on MacOS, but not under iOS 16: https://raw.githubusercontent.com/AOMediaCodec/av1-avif/master/testFiles/Netflix
The image should be viewed on an an HDR display, such as the XDR display on the 2021 M1 MacBook Pro.

Having support would be enormously beneficial, as there are now many millions iPhones and iPads which have HDR displays - but there is no valid way to view HDR images on a web browser on any of these devices.

[Karl Dubost]

Hi Greg, 

Thanks for the report.
Does it break Netflix? Or Netflix sends an alternate image for Safari?

[Greg]

Hi Karl-

I am not sure which assets they are using on their domain. This is just a handy reference image which is valid but will not render with WebKit (Safari and Chrome on iOS will not render this properly, but Chrome desktop will). My own personal use as as a content creator (photographer). I’ve generated and tested other HDR AVIF images which fail in a similar manner.

I have yet to find any iOS app to render an HDR image of any kind correctly. There is an odd exception to this. The camera roll will show proper HDR thumbnails (but not the full image, only its thumbnail), and only if the active image is not an HDR or the next image to the right of an HDR. So in this narrow context, I have seen HDR AVIF and EXR thumbnails properly show as bright images in the filmstrip.  But never in any mobile browser. 

-Greg

[Said Abou-Hallawa]

The link https://raw.githubusercontent.com/AOMediaCodec/av1-avif/master/testFiles/Netflix is broken. Can you attach an HDR AVIF image or put another working link?

[Greg]

Here's an alternative HDR AVIF: https://drive.google.com/file/d/1S1zHnz6oNKn_DEPaDQ1_VjGNh8p6Y-z-/view?usp=sharing

[Radar WebKit Bug Importer]

<rdar://problem/100879401>

[Myles C. Maxfield]

Is there something specific about AVIF that this bug is about, or is this bug about clipping on _any_ HDR image?

[Greg]

Probably HDR in general. I have not tested the experimental JXL build (nor looked for other possible formats which might be valid for HDR and supported by the browser).

[Greg]

ignore the JXL comment, was thinking of FireFox. I don't know if there's any public test for JXL in WebKit. Regardless, I have not tested other HDR formats beyond AVIF in Safari.

[Greg]

Here's a site with several HDR AVIF images for testing (renders properly in Chrome/Brave):
https://gregbenzphotography.com/hdr/

[Christopher Cameron]

The following test program shows the core issue:
https://github.com/ccameron-chromium/test-programs/blob/main/hlg-tone-map-simple.mm

That test program has 3 gradients:
- a CAMetalLayer that applies the HLG toLinear function
- a CAMetalLayer that applies to HLG toLinear function and sets its CAEDRMetalLayer to HLG
- an AVSampleBufferDisplayLayer that is HLG

The bottom two gradients looks the same, but the top one ramps up too quickly and then saturates.

In the top one, we're just applying the HLG inverse OETF directly to the pixel values, giving us values in the range [0, 12]. The HDR-to-SDR ratio of the display is not taken into account, so it will clip wherever we hit the EDR limit (pixel value 0.5 on SDR screens). This is what it appears is happening with HLG images in STP.

In the middle one, we're applying the HLG inverse OETF, and then setting CAEDRMetadata -- this uses the full display range.

In the bottom one, we tell AVSampleBufferDisplayLayer "do your thing", and it does the same thing as the middle layer.

The difficulty here is that when CoreGraphics is drawing HLG images, it needs to "bake in" the tone mapping that would ordinarily be applied by CoreAnimation. This also means that if the viewing conditions change, the image must be re-drawn using CG.

Another test page for this is:
https://ccameron-chromium.github.io/hdr-jpeg/index.html

[Xin Liu]

Here is another AVIF HDR test page:

https://people.csail.mit.edu/ericchan/hdr/hdr.php

It seems to be working on my iPhone 13 Pro Max with iOS 16 but not macOS Ventura Safari 16.

[Greg]

FireFox on iOS will render the images at https://people.csail.mit.edu/ericchan/hdr/hdr.php without being extremely dark, but they are still rendered incorrectly.

Open that URL in Chrome on MacOS and take a look at at clouds in the B&W mountain shot or the sunlit peaks in the shot just below it. The HDR detail isn't properly managed in any browser I've tested on iOS (including Chrome iOS).

[Greg]

Here's a test page with images encoded with ACR v15.1: https://gregbenzphotography.com/hdr-gallery/

If viewed on an HDR monitor, the page will render HDR AVIF images (if viewed on an SDR monitor, it will serve SDR JPG, but you can click the green button towards the bottom left to force it to render the HDR AVIF). These images include new metadata that help further improve rendering of HDR images. When viewed with the latest Chrome Dev or Chrome Canary, they show a notable improvement even over rendering in browser like Chrome that already supports HDR. This is a separate issue tracked in https://bugzilla.mozilla.org/show_bug.cgi?id=1805436.

[Said Abou-Hallawa]

Pull request: https://github.com/WebKit/WebKit/pull/10105

[EWS]

Committed 260512@main (2ba4ecb0408c): <https://commits.webkit.org/260512@main>

Reviewed commits have been landed. Closing PR #10105 and removing active labels.

[Rado]

Safari still doesn't display HDR AVIF, while Chrome supports it properly. This issue seems to be closed, should I open a new one? Thanks.

[Greg]

Agreed, this is an important issue to address. Lack of support in WebKit means no ability to properly see HDR photos when browsing with iPhone / iPad (these same devices look great with the HDR image support in the iOS 17 Photos app). And while Apple computer owners can work around this by using several alternatives to Safari, it is also not possible to easily see HDR photos in an Apple retail store either (which makes it hard to appreciate the benefit of XDR displays).


Issues created for missing HDR support in WebKit:
HDR AVIF: https://bugs.webkit.org/show_bug.cgi?id=265796
HDDR JPG gain maps: https://bugs.webkit.org/show_bug.cgi?id=265797