WebKit is incorrectly selecting a picture source it does not support

• Component: CSS
• Status: RESOLVED DUPLICATE

[Oliver Hunt]

On https://xeiaso.net/talks/how-my-website-works there are a series of picture elements, including avif and webp sources, and a png <img> fallback.

WebKit is providing a broken image icon as it is deciding that no option works, or it is erroneously pulling a source it doesn't support.

The relevant markup is:

<picture style="margin:0"><source type="image/avif" srcset="https://cdn.xeiaso.net/file/christine-static/talks/my-website/001.avif"><source type="image/webp" srcset="https://cdn.xeiaso.net/file/christine-static/talks/my-website/001.webp"><img style="padding:0" loading="lazy" src="https://cdn.xeiaso.net/file/christine-static/talks/my-website/001-smol.png"></picture>

[Oliver Hunt]

replacing the image/avif srcset with the png url results in a correct render.

[Alexey Proskuryakov]

In my testing, Safari 15.6.1 on macOS 12.5.1 works fine; Safari on iOS 16 doesn't.

[Radar WebKit Bug Importer]

<rdar://problem/99843785>

[Alexey Proskuryakov]

We support image/avif, so not sure if the diagnosis is correct. Seems like an issue with supporting these particular images, not an issue with picture fallback.

[Xe Iaso]

Hi, I posted some more details in case it helps on my blog: https://xeiaso.net/blog/avif-help-requested

Earlier today I pushed a commit to disable the AVIF files being included in the srcset for the picture element, but I reverted that so you can see the bug in action. If I am generating the AVIF pictures wrong, I would appreciate guidance on how to do so correctly so that I can create correct image files in the future. The part that confuses me is that Chrome accepts these AVIF images.

[Sam Sneddon [:gsnedders]]

(In reply to Alexey Proskuryakov from comment #4)
> We support image/avif, so not sure if the diagnosis is correct. Seems like
> an issue with supporting these particular images, not an issue with picture
> fallback.

Preview renders it correctly, which suggests that the problem isn't in ImageIO, but probably something else, like how we select image sources from picture elements.

[Simon Fraser (smfr)]

Does this reproduce without the "loading=lazy"?

[Markus Cadonau]

(In reply to Simon Fraser (smfr) from comment #7)
> Does this reproduce without the "loading=lazy"?

Yes, the portrait on my About page also appears broken: https://www.cadonau.net/about/

As well as the first two album covers, which are not lazy loaded, of my Photos page (along with the rest): https://www.cadonau.net/photos/

[Said Abou-Hallawa]

If you open https://cdn.xeiaso.net/file/christine-static/talks/my-website/001.avif in Preview and you show the inspector for the Tools menu, you will see that the format of this is image is HEIF image.

When calling the function CGImageSourceGetType() for this image, we get "public.heif". Since we are not supporting HEIF right now, WebKit fails to render this image.

I think the reason for returning "public.heif" is this image is a HEIF container which contains an AVIF image. So it is hard to argue CGImageSourceGetType() returns the wrong UTI.

The fix is either to make CGImageSourceGetType() return "public.avif" for this image or to enable HEIF/HEIC in WebKit.

[Said Abou-Hallawa]

The same problem is also mentioned in bug 244103.

[Said Abou-Hallawa]

Enabling HEIF/HEIC is tracked by bug 244424.

[Sam Sneddon [:gsnedders]]

Note that there's another issue here:

https://gpac.github.io/ComplianceWarden-wasm/avif.html points out:

[avif][Rule #7] Section 2.2.1
The values of the fields in the AV1CodecConfigurationBox shall match those of the Sequence Header OBU in the AV1 Image Item Data.

Currently neither libavif nor mp4parse-rust enforce this, but this would be another reason for this image to get rejected.

[Sam Sneddon [:gsnedders]]

(In reply to Said Abou-Hallawa from comment #9)
> If you open
> https://cdn.xeiaso.net/file/christine-static/talks/my-website/001.avif in
> Preview and you show the inspector for the Tools menu, you will see that the
> format of this is image is HEIF image.
> 
> When calling the function CGImageSourceGetType() for this image, we get
> "public.heif". Since we are not supporting HEIF right now, WebKit fails to
> render this image.

But we do support HEIF now, hence why we can render any AVIF image (which is, after all, AV1 in HEIF).

> I think the reason for returning "public.heif" is this image is a HEIF
> container which contains an AVIF image. So it is hard to argue
> CGImageSourceGetType() returns the wrong UTI.
> 
> The fix is either to make CGImageSourceGetType() return "public.avif" for
> this image or to enable HEIF/HEIC in WebKit.

This would just make WebKit wrong for other HEIF images, no? We currently support AVIF (AV1 in HEIF), bug 244424 is about adding support for HEIC (HEVC in HEIF), but it's at least also defined how to package JPEG, AVC, JPEG2000, and others in HEIF, plus likely other future types within the HEIF container.

Is your proposal that WebKit should allow any possible contents within an HEIF container, and we should add support for all defined payloads of the container format?

[Xe Iaso]

I found an error in the AVIF library I was using and have submitted a pull request to fix it: https://github.com/Kagami/go-avif/pull/17

It turns out the AVIF library I was using did set the AVIF images as HEIC images. I have left a broken image here for testing: https://cdn.xeiaso.net/file/christine-static/hero/happy-fireworks-pagodas.avif (this image is generated by AI and I disclaim any copyright to it, please feel free to add it to unit tests should you need to).

I have re-encoded every slide on the Oliver's link to be properly formatted AVIF files.

[Sam Sneddon [:gsnedders]]

(In reply to Xe Iaso from comment #14)
> I found an error in the AVIF library I was using and have submitted a pull
> request to fix it: https://github.com/Kagami/go-avif/pull/17
> 
> It turns out the AVIF library I was using did set the AVIF images as HEIC
> images.

This isn't setting them as HEIC images, this is setting the major brand as "generic image" without specifying what type it is. This should, per the AVIF spec, be absolutely fine.

[Said Abou-Hallawa]

https://cdn.xeiaso.net/file/christine-static/hero/happy-fireworks-pagodas.avif which used to be a broken image can be rendered correctly now in WebKit trunk.

So this bug can be resolved a dupe of bug 244103.

*** This bug has been marked as a duplicate of bug 244103 ***