Bug 245097 - WebKit is incorrectly selecting a picture source it does not support
Summary: WebKit is incorrectly selecting a picture source it does not support
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: CSS (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: BrowserCompat, InRadar
Depends on:
Blocks:
 
Reported: 2022-09-12 13:49 PDT by Oliver Hunt
Modified: 2022-09-16 11:13 PDT (History)
7 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Oliver Hunt 2022-09-12 13:49:40 PDT
On https://xeiaso.net/talks/how-my-website-works there are a series of picture elements, including avif and webp sources, and a png <img> fallback.

WebKit is providing a broken image icon as it is deciding that no option works, or it is erroneously pulling a source it doesn't support.

The relevant markup is:

<picture style="margin:0"><source type="image/avif" srcset="https://cdn.xeiaso.net/file/christine-static/talks/my-website/001.avif"><source type="image/webp" srcset="https://cdn.xeiaso.net/file/christine-static/talks/my-website/001.webp"><img style="padding:0" loading="lazy" src="https://cdn.xeiaso.net/file/christine-static/talks/my-website/001-smol.png"></picture>
Comment 1 Oliver Hunt 2022-09-12 13:51:33 PDT
replacing the image/avif srcset with the png url results in a correct render.
Comment 2 Alexey Proskuryakov 2022-09-12 16:35:32 PDT
In my testing, Safari 15.6.1 on macOS 12.5.1 works fine; Safari on iOS 16 doesn't.
Comment 3 Radar WebKit Bug Importer 2022-09-12 16:35:46 PDT
<rdar://problem/99843785>
Comment 4 Alexey Proskuryakov 2022-09-12 16:38:53 PDT
We support image/avif, so not sure if the diagnosis is correct. Seems like an issue with supporting these particular images, not an issue with picture fallback.
Comment 5 Xe Iaso 2022-09-12 19:00:00 PDT
Hi, I posted some more details in case it helps on my blog: https://xeiaso.net/blog/avif-help-requested

Earlier today I pushed a commit to disable the AVIF files being included in the srcset for the picture element, but I reverted that so you can see the bug in action. If I am generating the AVIF pictures wrong, I would appreciate guidance on how to do so correctly so that I can create correct image files in the future. The part that confuses me is that Chrome accepts these AVIF images.
Comment 6 Sam Sneddon [:gsnedders] 2022-09-13 09:24:50 PDT
(In reply to Alexey Proskuryakov from comment #4)
> We support image/avif, so not sure if the diagnosis is correct. Seems like
> an issue with supporting these particular images, not an issue with picture
> fallback.

Preview renders it correctly, which suggests that the problem isn't in ImageIO, but probably something else, like how we select image sources from picture elements.
Comment 7 Simon Fraser (smfr) 2022-09-13 15:11:02 PDT
Does this reproduce without the "loading=lazy"?
Comment 8 Markus Cadonau 2022-09-14 00:41:32 PDT
(In reply to Simon Fraser (smfr) from comment #7)
> Does this reproduce without the "loading=lazy"?

Yes, the portrait on my About page also appears broken: https://www.cadonau.net/about/

As well as the first two album covers, which are not lazy loaded, of my Photos page (along with the rest): https://www.cadonau.net/photos/
Comment 9 Said Abou-Hallawa 2022-09-14 22:09:55 PDT
If you open https://cdn.xeiaso.net/file/christine-static/talks/my-website/001.avif in Preview and you show the inspector for the Tools menu, you will see that the format of this is image is HEIF image.

When calling the function CGImageSourceGetType() for this image, we get "public.heif". Since we are not supporting HEIF right now, WebKit fails to render this image.

I think the reason for returning "public.heif" is this image is a HEIF container which contains an AVIF image. So it is hard to argue CGImageSourceGetType() returns the wrong UTI.

The fix is either to make CGImageSourceGetType() return "public.avif" for this image or to enable HEIF/HEIC in WebKit.
Comment 10 Said Abou-Hallawa 2022-09-14 22:11:22 PDT
The same problem is also mentioned in bug 244103.
Comment 11 Said Abou-Hallawa 2022-09-14 22:20:35 PDT
Enabling HEIF/HEIC is tracked by bug 244424.
Comment 12 Sam Sneddon [:gsnedders] 2022-09-15 14:06:00 PDT
Note that there's another issue here:

https://gpac.github.io/ComplianceWarden-wasm/avif.html points out:

[avif][Rule #7] Section 2.2.1
The values of the fields in the AV1CodecConfigurationBox shall match those of the Sequence Header OBU in the AV1 Image Item Data.

Currently neither libavif nor mp4parse-rust enforce this, but this would be another reason for this image to get rejected.
Comment 13 Sam Sneddon [:gsnedders] 2022-09-15 14:21:56 PDT
(In reply to Said Abou-Hallawa from comment #9)
> If you open
> https://cdn.xeiaso.net/file/christine-static/talks/my-website/001.avif in
> Preview and you show the inspector for the Tools menu, you will see that the
> format of this is image is HEIF image.
> 
> When calling the function CGImageSourceGetType() for this image, we get
> "public.heif". Since we are not supporting HEIF right now, WebKit fails to
> render this image.

But we do support HEIF now, hence why we can render any AVIF image (which is, after all, AV1 in HEIF).

> I think the reason for returning "public.heif" is this image is a HEIF
> container which contains an AVIF image. So it is hard to argue
> CGImageSourceGetType() returns the wrong UTI.
> 
> The fix is either to make CGImageSourceGetType() return "public.avif" for
> this image or to enable HEIF/HEIC in WebKit.

This would just make WebKit wrong for other HEIF images, no? We currently support AVIF (AV1 in HEIF), bug 244424 is about adding support for HEIC (HEVC in HEIF), but it's at least also defined how to package JPEG, AVC, JPEG2000, and others in HEIF, plus likely other future types within the HEIF container.

Is your proposal that WebKit should allow any possible contents within an HEIF container, and we should add support for all defined payloads of the container format?
Comment 14 Xe Iaso 2022-09-16 05:29:16 PDT
I found an error in the AVIF library I was using and have submitted a pull request to fix it: https://github.com/Kagami/go-avif/pull/17

It turns out the AVIF library I was using did set the AVIF images as HEIC images. I have left a broken image here for testing: https://cdn.xeiaso.net/file/christine-static/hero/happy-fireworks-pagodas.avif (this image is generated by AI and I disclaim any copyright to it, please feel free to add it to unit tests should you need to).

I have re-encoded every slide on the Oliver's link to be properly formatted AVIF files.
Comment 15 Sam Sneddon [:gsnedders] 2022-09-16 11:13:44 PDT
(In reply to Xe Iaso from comment #14)
> I found an error in the AVIF library I was using and have submitted a pull
> request to fix it: https://github.com/Kagami/go-avif/pull/17
> 
> It turns out the AVIF library I was using did set the AVIF images as HEIC
> images.

This isn't setting them as HEIC images, this is setting the major brand as "generic image" without specifying what type it is. This should, per the AVIF spec, be absolutely fine.