Bug 236362 - [Flatpak SDK] Add mold linker
Summary: [Flatpak SDK] Add mold linker
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Local Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Adrian Perez
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2022-02-09 06:26 PST by Adrian Perez
Modified: 2022-05-06 06:50 PDT (History)
7 users (show)

See Also:


Attachments
Patch (2.64 KB, patch)
2022-02-09 06:38 PST, Adrian Perez
no flags Details | Formatted Diff | Diff
Patch v2 (2.66 KB, patch)
2022-02-10 05:52 PST, Adrian Perez
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Adrian Perez 2022-02-09 06:26:18 PST
I have it working locally and intend to do some benchmarks to see how
much faster we can get debug (developer) builds by switching to ld.lld
or ld.mold

For those who may not yet know about it, you can learn about Mold here:

  https://github.com/rui314/mold

While it is still missing a few features, the only important issue is
that Mold does *not* support LTO at the moment:

  https://github.com/rui314/mold/issues/181

Other than this, Mold is perfectly usable today for complex projects
like WebKit, and given that developers typically do not do LTO builds
when working on the codebase, it may already be interesting to have it
in the SDK for those.
Comment 1 Adrian Perez 2022-02-09 06:38:11 PST
Created attachment 451367 [details]
Patch
Comment 2 Adrian Perez 2022-02-09 07:11:37 PST
To use Mold, either of these work, tested both with the WPE and GTK ports:

GCC ≥ 12:
  % LDFLAGS=-fuse-ld=mold Tools/Scripts/build-webkit [...]

GCC < 12:
  % LDFLAGS=-B/usr/lib/mold Tools/Scripts/build-webkit [...]

Clang:
  % LDFLAGS=-fuse-ld=mold CC=clang CXX=clang++ Tools/Scripts/build-webkit [...]

Adding “-flto” to {C,CXX,LD}FLAGS will result in linker failures due to
the missing support for LTO in Mold.
Comment 3 Philippe Normand 2022-02-09 07:34:54 PST
Comment on attachment 451367 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=451367&action=review

> Tools/buildstream/elements/sdk/mold.bst:12
> +- kind: tar
> +  url: https://github.com/rui314/mold/archive/refs/tags/v1.0.3.tar.gz

We usually track git tags instead of release tarballs, at least for projects hosted github. Any special reason to do it differently here?
Comment 4 Adrian Perez 2022-02-09 07:39:08 PST
(In reply to Philippe Normand from comment #3)
> Comment on attachment 451367 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=451367&action=review
> 
> > Tools/buildstream/elements/sdk/mold.bst:12
> > +- kind: tar
> > +  url: https://github.com/rui314/mold/archive/refs/tags/v1.0.3.tar.gz
> 
> We usually track git tags instead of release tarballs, at least for projects
> hosted github. Any special reason to do it differently here?

Fetching Git repositories when there are tarballs available is wasteful
as it uses more bandwidth and local disk space. It may seem moot, but it
all adds up.
Comment 5 Adrian Perez 2022-02-09 13:25:20 PST
I had a chat with Philippe and it turns out that referencing Git commits
is what is usually done in the Freedesktop SDK sources — I still wonder
why it's done like that, but for consistency I will change the patch to
use a Git ref.
Comment 6 Adrian Perez 2022-02-09 15:05:38 PST
(In reply to Adrian Perez from comment #2)
 
> GCC < 12:
>   % LDFLAGS=-B/usr/lib/mold Tools/Scripts/build-webkit [...]

This is actually:

  % LDFLAGS=-B/usr/libexec/mold Tools/Scripts/build-webkit [...]

(In Arch Linux there is no “libexec/” but the Flatpak SDK has it,
that's where my confusion came from.)
Comment 7 Adrian Perez 2022-02-10 04:31:33 PST
I did seven runs with each linker, took out the outliers,
and got the following results:

 Linker |       Times         |  Avg. | Speedup
 -------+---------------------+-------+------------------
    BFD | 393 403 273 267 274 | 322.0 | 1.00x  (baseline)
   Mold | 117 106  99 107 105 | 106.8 | 3.01x
   Gold |  81 116  82  79  82 |  88.0 | 3.66x
    LLD | 106  70  76  70  72 |  78.8 | 4.09x

For each linker, I first removed the build directory, then ran
the build command to have all the object files built, and for
each run removed “bin/*” and “lib/*.so*” before to make sure
that the time differences measured are caused by switching the
linker. Commands used for each were:

  LDFLAGS=-B/usr/libexec/mold NUMBER_OF_PROCESSORS=33 \
    ./Tools/Scripts/build-webkit --gtk --debug --cmakeargs='-DUSE_LD_LLD=OFF'
  LDFLAGS=-fuse-ld=bfd NUMBER_OF_PROCESSORS=33 \
    ./Tools/Scripts/build-webkit --gtk --debug --cmakeargs='-DUSE_LD_LLD=OFF'
  LDFLAGS=-fuse-ld=gold NUMBER_OF_PROCESSORS=33 \
    ./Tools/Scripts/build-webkit --gtk --debug --cmakeargs='-DUSE_LD_LLD=OFF'
  LDFLAGS=-fuse-ld=lld NUMBER_OF_PROCESSORS=33 \
    ./Tools/Scripts/build-webkit --gtk --debug --cmakeargs='-DUSE_LD_LLD=ON'

This was done using the Flatpak SDK, which means the compiler was
GCC 11.2.0 -- hence the need to use the -B<path> flag to make it use
Mold. I don't know if this might be the cause of the Mold not being
faster but I would not expect that the compiler driver needing to do
program lookups in a different way would cause a slowdown; but it could
as well be an issue in Mold itself.

It would be interesting to use Clang as the compiler (and hence as
driver for the linking) so all linkers can be chosen in the same way
using the -fuse-ld=<name> flag.

Given these numbers, switching from the BFD linker to LLD by default
for debug/development builds should be a no brainer. We have bug #235979
for that =)
Comment 8 Adrian Perez 2022-02-10 05:52:19 PST
Created attachment 451521 [details]
Patch v2
Comment 9 Adrian Perez 2022-02-11 13:19:43 PST
(In reply to Adrian Perez from comment #0)

> While it is still missing a few features, the only important issue is
> that Mold does *not* support LTO at the moment:
> 
>   https://github.com/rui314/mold/issues/181

Initial LTO support just landed earlier today:

  https://github.com/rui314/mold/commit/46995bcfc3e3113133620bf16445c5f13cd76a18

\o/
Comment 10 Philippe Normand 2022-02-15 04:09:53 PST
Thanks! I can include this in the next SDK roll-out which I'm preparing already.
Comment 11 Adrian Perez 2022-02-15 04:11:25 PST
(In reply to Philippe Normand from comment #10)
> Thanks! I can include this in the next SDK roll-out which I'm preparing
> already.

Sounds good! 👍️
Comment 12 EWS 2022-02-15 04:32:41 PST
Committed r289801 (247266@main): <https://commits.webkit.org/247266@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 451521 [details].
Comment 13 Radar WebKit Bug Importer 2022-02-15 04:33:31 PST
<rdar://problem/88958643>