Bug 233326 - [Flatpak SDK] Add build recipe for libjxl
Summary: [Flatpak SDK] Add build recipe for libjxl
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Local Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Adrian Perez
URL:
Keywords: InRadar
Depends on:
Blocks: 233325
  Show dependency treegraph
 
Reported: 2021-11-18 12:18 PST by Adrian Perez
Modified: 2021-11-23 16:37 PST (History)
6 users (show)

See Also:


Attachments
Patch (4.81 KB, patch)
2021-11-18 14:50 PST, Adrian Perez
no flags Details | Formatted Diff | Diff
Patch (1.85 KB, patch)
2021-11-22 15:28 PST, Adrian Perez
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Adrian Perez 2021-11-18 12:18:48 PST
'tis needed for JPEG-XL support
Comment 1 Adrian Perez 2021-11-18 14:50:42 PST
Created attachment 444743 [details]
Patch
Comment 2 Adrian Perez 2021-11-19 05:48:02 PST
(In reply to Adrian Perez from comment #1)
> Created attachment 444743 [details]
> Patch

I built the SDK locally and completed correctly =]
Comment 3 EWS 2021-11-19 06:39:16 PST
Committed r286057 (244444@main): <https://commits.webkit.org/244444@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 444743 [details].
Comment 4 Radar WebKit Bug Importer 2021-11-19 06:40:24 PST
<rdar://problem/85600724>
Comment 5 Philippe Normand 2021-11-20 03:24:29 PST
(In reply to Adrian Perez from comment #2)
> (In reply to Adrian Perez from comment #1)
> > Created attachment 444743 [details]
> > Patch
> 
> I built the SDK locally and completed correctly =]

Will you deploy it as well?
Comment 6 Adrian Perez 2021-11-22 12:49:44 PST
Reopening… the build system for libjxl gets confused by telling it to
use the system-installed libhwy (built by highway.bst) because it is a
static library used internally in the implementation of libjxl, yet it
adds it as a requirement into libjxl.pc, even when it should not be there
in that case:

  % webkit-flatpak --repo=$R -c pkg-config --cflags libjxl
  Package libhwy was not found in the pkg-config search path.
  Perhaps you should add the directory containing `libhwy.pc'
  to the PKG_CONFIG_PATH environment variable
  Package 'libhwy', required by 'libjxl', not found
  %

It seems that the assumption is that using -DJPEGXL_FORCE_SYSTEM_HWY=ON
then the system-provided libhighway *has* to be a dynamic library and
that is why it gets added as a reqirement in libjxl.pc — I cannot find
exactly how that is done, but it looks like some internal CMake sorcery.

I am going to try a follow-up where we place the libhighway sources into
the third_party/highway/ subdir, like we already do for SkCMS and LodePNG,
and let the libjxl CMake files build it statically itself 🤷
Comment 7 Adrian Perez 2021-11-22 15:28:24 PST
Created attachment 444987 [details]
Patch
Comment 8 EWS 2021-11-23 00:54:13 PST
Committed r286127 (244514@main): <https://commits.webkit.org/244514@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 444987 [details].
Comment 9 Adrian Perez 2021-11-23 03:13:21 PST
(In reply to Philippe Normand from comment #5)
> (In reply to Adrian Perez from comment #2)
> > (In reply to Adrian Perez from comment #1)
> > > Created attachment 444743 [details]
> > > Patch
> > 
> > I built the SDK locally and completed correctly =]
> 
> Will you deploy it as well?

Done!
Comment 10 Michael Catanzaro 2021-11-23 10:40:45 PST
There are still EWS failures, e.g. https://ews-build.webkit.org/#/builders/36/builds/52314. Looks like the EWS is having trouble:

perl Tools/Scripts/update-webkitgtk-libs --release
 in dir /home/ews/worker/GTK-Build-EWS/build (timeout 1200 secs)
 watching logfiles {}
 argv: ['perl', 'Tools/Scripts/update-webkitgtk-libs', '--release']
 using PTY: False
Updating Flatpak Release environment
Looking for updates…
F: Warning: Treating remote fetch error as non-fatal since runtime/org.webkit.Sdk/x86_64/21.08 is already installed: Unable to load summary from remote webkit-sdk: Unacceptable TLS certificate
F: Warning: Can't find runtime/org.webkit.Sdk/x86_64/21.08 metadata for dependencies: Unable to load metadata from remote webkit-sdk: summary fetch error: Unacceptable TLS certificate
F: Warning: Treating remote fetch error as non-fatal since runtime/org.webkit.Sdk.Locale/x86_64/21.08 is already installed: Unable to load summary from remote webkit-sdk: Unacceptable TLS certificate
F: Warning: Can't find runtime/org.webkit.Sdk.Locale/x86_64/21.08 metadata for dependencies: Unable to load metadata from remote webkit-sdk: summary fetch error: Unacceptable TLS certificate
F: Warning: Treating remote fetch error as non-fatal since runtime/org.webkit.Sdk.Debug/x86_64/21.08 is already installed: Unable to load summary from remote webkit-sdk: Unacceptable TLS certificate
F: Warning: Can't find runtime/org.webkit.Sdk.Debug/x86_64/21.08 metadata for dependencies: Unable to load metadata from remote webkit-sdk: summary fetch error: Unacceptable TLS certificate
F: Warning: Treating remote fetch error as non-fatal since runtime/org.webkit.Platform/x86_64/21.08 is already installed: Unable to load summary from remote webkit-sdk: Unacceptable TLS certificate
F: Warning: Can't find runtime/org.webkit.Platform/x86_64/21.08 metadata for dependencies: Unable to load metadata from remote webkit-sdk: summary fetch error: Unacceptable TLS certificate
F: Warning: Treating remote fetch error as non-fatal since runtime/org.webkit.Platform.Locale/x86_64/21.08 is already installed: Unable to load summary from remote webkit-sdk: Unacceptable TLS certificate
F: Warning: Can't find runtime/org.webkit.Platform.Locale/x86_64/21.08 metadata for dependencies: Unable to load metadata from remote webkit-sdk: summary fetch error: Unacceptable TLS certificate
Nothing to do.
Comment 11 Adrian Perez 2021-11-23 12:13:48 PST
It's related to the recent Let's Encrypt certificate that was revoked,
which is preventing some bots from fetching an updated SDK — nothing
to do with libjxl itself. Right now I'm doing a sweep of all the bots
we maintain to make sure they can fetch content from hosts which use
LE certificates.
Comment 12 Michael Catanzaro 2021-11-23 16:07:39 PST
(In reply to Adrian Perez from comment #11)
> It's related to the recent Let's Encrypt certificate that was revoked

I doubt anything was revoked. Surely it must be https://letsencrypt.org/2021/10/01/cert-chaining-help.html?

Apparently WebKitGTK is super broken in Ubuntu 20.04 right now because -- amazingly -- Ubuntu never updated GnuTLS the last time this happened during summer 2020. It's just been broken for 1.5 years without anybody noticing, or at least without anybody bothering to report it to us. Just discovered that in https://gitlab.gnome.org/GNOME/glib-networking/-/issues/177#note_1311945. That's *very* concerning. But I think our EWS run on Debian, right, which should be unaffected because Debian was patched right away. And OpenSSL has been able to handle this chain since OpenSSL 1.1, which is pretty old. So it's hard to guess what is going wrong for the EWS tbh.
Comment 13 Adrian Perez 2021-11-23 16:37:35 PST
(In reply to Michael Catanzaro from comment #12)
> (In reply to Adrian Perez from comment #11)
> > It's related to the recent Let's Encrypt certificate that was revoked
> 
> I doubt anything was revoked. Surely it must be
> https://letsencrypt.org/2021/10/01/cert-chaining-help.html?

Revoked, chain changed... potato, potatoeh 🤷‍♂️️

> Apparently WebKitGTK is super broken in Ubuntu 20.04 right now because --
> amazingly -- Ubuntu never updated GnuTLS the last time this happened during
> summer 2020. It's just been broken for 1.5 years without anybody noticing,
> or at least without anybody bothering to report it to us. Just discovered
> that in
> https://gitlab.gnome.org/GNOME/glib-networking/-/issues/177#note_1311945.
> That's *very* concerning.

Ouch.

> But I think our EWS run on Debian, right, which
> should be unaffected because Debian was patched right away. And OpenSSL has
> been able to handle this chain since OpenSSL 1.1, which is pretty old. So
> it's hard to guess what is going wrong for the EWS tbh.

They run Debian and that would be good if the patches would have been
installed (hint: guess of often bots get updated, given that we don't
care much about the system because we ship another full system in the
Flatpak SDK…)

Anyhoo, all is working now 🛠️