Bug 208235 - Support JPEG XL images
Summary: Support JPEG XL images
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Images (show other bugs)
Version: WebKit Nightly Build
Hardware: All All
: P2 Enhancement
Assignee: Nobody
URL:
Keywords: InRadar
: 252509 (view as bug list)
Depends on: 233113 233325 233364 233545 257871 258655
Blocks:
  Show dependency treegraph
 
Reported: 2020-02-26 04:51 PST by Gunther Brunner
Modified: 2023-06-28 19:18 PDT (History)
28 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Gunther Brunner 2020-02-26 04:51:33 PST
Why:
JPEG XL is the future of all image formats.

Explanation here:
https://www.slideshare.net/cloudinarymarketing/imagecon-2019-jon-sneyer

Reference implementation here:
https://gitlab.com/wg1/jpeg-xl
Comment 1 Jyrki Alakuijala 2021-04-23 06:34:54 PDT
I copied the description text from chromestatus.com https://www.chromestatus.com/feature/5188299478007808 here:

JPEG XL is a new royalty-free image codec targeting the image quality as found on the web, providing about 60% size savings when compared to original JPEG at the same perceptual quality, while supporting modern features like HDR, animation, alpha channel, lossless JPEG recompression, lossless and progressive modes. It is based on Google's PIK and Cloudinary's FUIF, and is in the final steps of standardization with ISO.

Motivation
The main motivations for supporting JPEG XL in Chrome are:
 - The improvement in image quality vs image size, about 60% file size savings for the same visual quality (lossy compression of larger originals) when compared to JPEG at the qualities found on the web.
Improved visual latency by both smaller download sizes and supporting progressive decoding modes.
 - Support for HDR, animation and progressive all together in the same image codec. 
 - Support for lossless-recompressed JPEGs
 - Ecosystem interest in JPEG XL: Several Google teams evaluated using JPEG XL for storing and delivering images, as well as outside of Google: including CDNs interest in storing lossless-recompressed JPEGs as JPEG XL and converting to JPEG on request is the browser doesn't support JXL. Facebook is exploring to use JPEG XL.
Comment 2 Jyrki Alakuijala 2021-04-23 06:44:45 PDT
Some more links about JPEG XL:

Committee draft of the JPEG XL spec: https://arxiv.org/abs/1908.03565

Search for JPEG XL issues on chromium.org: https://bugs.chromium.org/p/chromium/issues/list?q=component%3ABlink%3EImage%20jpeg%20xl&can=2

JPEG XL tracking bug on chromium: https://bugs.chromium.org/p/chromium/issues/detail?id=1178058

Intent-to-prototype chromium.org blink-dev group discussion thread: https://groups.google.com/a/chromium.org/g/blink-dev/c/WjCKcBw219k

https://www.chromestatus.com/features#tags%3Ajxl shows both the content encoding and image encoding possibilities for JPEG XL. After reflecting on the advice from W3C HTTP WG members <https://lists.w3.org/Archives/Public/ietf-http-wg/2020JulSep/0102.html> we are prioritizing image coding over content encoding.
Comment 3 Jyrki Alakuijala 2021-04-26 12:17:02 PDT
Respective Bugzilla entries:

Initial bug with some discussion: https://bugzilla.mozilla.org/show_bug.cgi?id=1539075

Initial implementation of JXL in Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=1707590
Comment 4 Sam Sneddon [:gsnedders] 2021-05-06 08:17:55 PDT
See also the thread at https://lists.webkit.org/pipermail/webkit-dev/2021-May/031844.html

As noted there, for the Apple WebKit ports, image format support depends on OS library support, and those libraries do not currently support JPEG XL.
Comment 5 Erik Andre 2021-05-26 01:59:07 PDT
Erik Andre from the Images Infra team at Facebook here. I'd like to share a bit of our view on JPEG XL in the context of new image formats (e.g AVIF, JPEG XL, WEBP2, ...) and how browser adoption will let us move forward with our plans to test and hopefully roll out JPEG XL.

After spending the last 5 months investigating and evaluating JPEG XL from both a performance and quality point of view, it's our opinion that JPEG XL has the most potential of the new generation of image formats that are trying to succeed JPEG.
This opinion is based on the following findings:

* JPEG XL encoding at speed/effort 6 is as fast as JPEG encoding (using MozJpeg with Trellis encoding enabled). This means that it's practical to encode JPEG XL images on the fly and serve to client. This can be compared with the encoding speed of AVIF which necessitates offline encoding which offers less flexibility when it comes to delivering dynamically sized and compressed content.
* Depending on the settings used, JPEG XL can also be very fast to decode. Our mobile benchmarks show that we can reach parity with JPEG when using multiple threads to decode. This matches and in many cases surpasses the decoding performance of other new image formats.
* The JPEG XL image format offers flexible support for progressive decoding, allowing quick preview to be rendered based on just the DC components, or multiple intermediate images without being limited to a fixed set of scans chosen at encode time (i.e progressive JPEG scans).

Having browser support from all the major browsers is going to make our lives a lot easier in upcoming WWW experiments and ensure that we can deliver a consistent experience across platforms and browsers.
Comment 8 Michael Catanzaro 2021-11-18 13:52:06 PST
Note there's been action in bug #233113 and bug #233325. This doesn't cover PlayStation or FTL Windows yet, but it's enough to make JPEG XL readily available to the non-Apple ports.

Question: does libjxl handle color management for us? I've never really understood what color management is tbh, but I know we have X11-specific LCMS integration that's required in PNGImageDecoder.cpp and JPEGImageDecoder.cpp, and I wonder if libjxl handles that for us, or if it's just not required with JPEG XL? (I'm not pleased that we added PNG and JPEG color management that's incompatible with Wayland. That wasn't forward-thinking.)

(In reply to Erik Andre from comment #5)
> After spending the last 5 months investigating and evaluating JPEG XL from
> both a performance and quality point of view, it's our opinion that JPEG XL
> has the most potential of the new generation of image formats that are
> trying to succeed JPEG.

Just wanted to let you know your comment helped me understand the interest in JPEG XL. I remember JPEG 2000 and was keen to avoid a repeat (we are probably stuck supporting JPEG 2000 indefinitely).
Comment 9 Jon Sneyers 2021-11-19 01:02:38 PST
No, libjxl cannot handle color management for you, for the simple reason that libjxl has no way of knowing the display color space. So it is the same as with any other image format in that respect: you get uncompressed pixel values and an ICC profile, which together define a decoded image. To show those pixels on a screen, you have to convert them from the image color space (given by the ICC profile) to the display color space (which is how the screen will interpret pixel values). You cannot simply assume that the display color space will be sRGB; wider gamut screens are common (e.g. Display P3), and also HDR screens are becoming more common (e.g. Rec.2100 PQ or HLG).

That said; if you happen to know the display color space, using JxlDecoderSetPreferredColorProfile() you _can_ ask the libjxl decoder to try to do the color management for you, i.e. to return pixels that are already in that color space. This is only possible for certain display color spaces (the ones that can be described by a JxlColorEncoding, which includes most color spaces used in practice like sRGB, AdobeRGB, Display P3, DCI-P3, Rec.2100, etc; it doesn't include more complicated color spaces though that e.g. use lookup tables), and even then it is not guaranteed that libjxl will be able to return pixels in the requested space: it can only do that in the case of lossy images (which use an internal absolute color space called XYB) and in the case of lossless images that are in a JxlColorEncoding color space; it cannot do that in more complicated cases like CMYK or weird arbitrary-ICC-profile cases.

Anyway, in practice you should just take the pixel data and the ICC profile returned by the libjxl decoder and do the same thing with it as with any other image. It can be optimized a bit in case you happen to know the display space - using JxlDecoderSetPreferredColorProfile() you can make it very likely that the returned ICC profile happens to be identical to the display space, so your CMS gets a no-op case (pixel values don't need to be converted at all) - but that's an optimization I wouldn't bother to implement in an initial implementation of jxl support (since likely it requires quite a bit of nontrivial code plumbing for you to actually know the display space).
Comment 10 Michael Catanzaro 2021-11-19 06:05:00 PST
(In reply to Jon Sneyers from comment #9)
> Anyway, in practice you should just take the pixel data and the ICC profile
> returned by the libjxl decoder and do the same thing with it as with any
> other image.

Very interesting. Currently WebKit image decoders use ICC profiles only for PNG and JPEG, not for any other image format. And it does so only when running on X11, not on Wayland or Windows or any other platform. (Reminder: Apple ports do not use WebKit image decoders.) Sounds like we have some room for improvement there....
Comment 11 Michael Catanzaro 2021-11-19 06:18:26 PST
(In reply to Michael Catanzaro from comment #10)
> Very interesting. Currently WebKit image decoders use ICC profiles only for
> PNG and JPEG, not for any other image format. And it does so only when
> running on X11, not on Wayland or Windows or any other platform. (Reminder:
> Apple ports do not use WebKit image decoders.) Sounds like we have some room
> for improvement there....

Reported: bug #233364.
Comment 13 Brent Fulgham 2022-07-01 15:21:29 PDT
This is tracked in the relevant operating system component in this radar:
<rdar://77607260>

However, it does not seem that Chrome or Firefox plan to ship this support.
Comment 14 Brent Fulgham 2022-07-01 15:21:35 PDT
This is tracked in the relevant operating system component in this radar:
<rdar://77607260>

However, it does not seem that Chrome or Firefox plan to ship this support.
Comment 15 Greg 2022-09-29 19:07:56 PDT
It will be especially important for JPEG XL (JXL) support to also properly handle HDR (10 and 12-bit) images. Legacy file formats are not suitable for HDR. AVIF is likely to have a head start, but JXL is a critical file format to support for HDR given faster encoding and smaller files.

The majority of Apple displays in the past several years have HDR display hardware (https://support.apple.com/en-us/HT210980). This is collectively hundreds of millions of devices now, and the XDR displays (new MacBook Pros, iPhones, etc) are absolutely incredible. However, the only way to view HDR images on the web on these gorgeous screens is with Chrome on MacOS. Safari and WebKit do not support, which means that none of the iOS mobile devices have any support (including via Chrome on iOS given WebKit dependency).
Comment 16 Jyrki Alakuijala 2022-09-30 05:55:10 PDT
JPEG XL stores images in XYB, in absolute color (nits). This makes the storage and parametrization of SDR and HDR the same in JPEG XL encoding and decoding. HDR specialities are only needed in interface code, for tone-mapping the images for lower dynamic range displays.
Comment 17 Greg 2022-09-30 06:27:06 PDT
Jyrki-  Good to know. AVIF must be encoded differently (it currently renders as clipped to SDR on iOS16).
Comment 18 Ahmad Saleem 2023-02-17 18:10:11 PST
*** Bug 252509 has been marked as a duplicate of this bug. ***
Comment 19 Sam Sneddon [:gsnedders] 2023-02-20 15:35:20 PST
(In reply to Greg from comment #17)
> Jyrki-  Good to know. AVIF must be encoded differently (it currently renders
> as clipped to SDR on iOS16).

see Bug 245858
Comment 20 Michael Catanzaro 2023-06-08 15:35:27 PDT
All dependent bugs are closed. \o/
Comment 21 Michael Catanzaro 2023-06-08 15:42:17 PDT
Well, sometimes I comment too soon. I've reported bug #208235 to enable JPEG XL by default, which will land soon. No need to keep this bug open regardless.