Bug 219485 - [JSC] Bundle non-native libs in run-jsc-stress-tests
Summary: [JSC] Bundle non-native libs in run-jsc-stress-tests
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-12-03 02:49 PST by Angelos Oikonomopoulos
Modified: 2023-09-07 13:48 PDT (History)
10 users (show)

See Also:


Attachments
Patch (16.25 KB, patch)
2020-12-03 02:57 PST, Angelos Oikonomopoulos
no flags Details | Formatted Diff | Diff
Patch (16.36 KB, patch)
2020-12-03 06:36 PST, Angelos Oikonomopoulos
no flags Details | Formatted Diff | Diff
Patch (16.32 KB, patch)
2020-12-04 01:56 PST, Angelos Oikonomopoulos
no flags Details | Formatted Diff | Diff
Patch (16.32 KB, patch)
2020-12-04 05:19 PST, Angelos Oikonomopoulos
no flags Details | Formatted Diff | Diff
Shell script to find dependencies using readelf (1.86 KB, application/x-shellscript)
2020-12-07 10:15 PST, Adrian Perez
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Angelos Oikonomopoulos 2020-12-03 02:49:10 PST
[JSC] Bundle non-native libs in run-jsc-stress-tests
Comment 1 Angelos Oikonomopoulos 2020-12-03 02:57:18 PST
Created attachment 415289 [details]
Patch
Comment 2 Carlos Alberto Lopez Perez 2020-12-03 06:11:06 PST
Comment on attachment 415289 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=415289&action=review

> Tools/Scripts/run-jsc-stress-tests:1886
> +                        "--ldd=#{$ldd}",

I guess we should only pass the "-ldd" parameter here if we were called with it
Comment 3 Angelos Oikonomopoulos 2020-12-03 06:36:02 PST
Created attachment 415301 [details]
Patch
Comment 4 Angelos Oikonomopoulos 2020-12-03 06:36:39 PST
(In reply to Carlos Alberto Lopez Perez from comment #2)
> Comment on attachment 415289 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=415289&action=review
> 
> > Tools/Scripts/run-jsc-stress-tests:1886
> > +                        "--ldd=#{$ldd}",
> 
> I guess we should only pass the "-ldd" parameter here if we were called with
> it

Makes sense. Fixed.
Comment 5 Angelos Oikonomopoulos 2020-12-04 01:56:14 PST
Created attachment 415396 [details]
Patch
Comment 6 Angelos Oikonomopoulos 2020-12-04 05:19:52 PST
Created attachment 415409 [details]
Patch
Comment 7 EWS 2020-12-07 01:38:44 PST
Committed r270497: <https://trac.webkit.org/changeset/270497>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 415409 [details].
Comment 8 Radar WebKit Bug Importer 2020-12-07 01:39:16 PST
<rdar://problem/72039953>
Comment 9 Adrian Perez 2020-12-07 04:19:12 PST
Comment on attachment 415409 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=415409&action=review

> Tools/Scripts/generate-bundle:122
> +    def __init__(self, configuration, platform, bundle_type, syslibs, ldd, should_strip_objects, compression_type, destination = None, revision = None, builder_name = None):

We could avoid needing to pass by hand some alternative “ldd” tool
by always using ”readelf”, which works for any ELF file (no matter
the target architecture) and does not try to run any code. That
would make this “ldd” parameter to the BundleCreator unnecessary.

Something like this gets you the list of shared objects needed:

   readelf -Wd <file> | \
     awk '$2 == "(NEEDED)" { print substr($5, 2, length($5)-2) }'

The output will only include direct dependencies of “<file>”, so one
needs to run the same command recursively on each listed object, until
no more objects are added to the set. I have some shell script around
that does just that and also takes into account the DT_RUNPATH entries
from the ELF dynamic section—let me know if you are interested and I
can share it.
Comment 10 Angelos Oikonomopoulos 2020-12-07 04:22:11 PST
(In reply to Adrian Perez from comment #9)
> Comment on attachment 415409 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=415409&action=review
> 
> > Tools/Scripts/generate-bundle:122
> > +    def __init__(self, configuration, platform, bundle_type, syslibs, ldd, should_strip_objects, compression_type, destination = None, revision = None, builder_name = None):
> 
> We could avoid needing to pass by hand some alternative “ldd” tool
> by always using ”readelf”, which works for any ELF file (no matter
> the target architecture) and does not try to run any code. That
> would make this “ldd” parameter to the BundleCreator unnecessary.

That would make a number of things simpler, yah. Was not aware readelf had this functionality :/
Comment 11 Adrian Perez 2020-12-07 10:15:50 PST
Created attachment 415562 [details]
Shell script to find dependencies using readelf

Here's a standalone version of the script I mentioned :)
Comment 12 Loïc Yhuel 2023-08-07 05:30:48 PDT
The commit mentions "xldd from crosstool-NG", but this one outputs unprefixed paths (no CT_XLDD_ROOT or sysroot), so it doesn't work.

It seems the intended script was https://trac.webkit.org/attachment/wiki/JSCOnly/CrossBuildAndRemoteTestJSCLinux/cross-ldd.
Comment 13 Angelos Oikonomopoulos 2023-09-07 13:48:44 PDT
(In reply to Loïc Yhuel from comment #12)
> The commit mentions "xldd from crosstool-NG", but this one outputs
> unprefixed paths (no CT_XLDD_ROOT or sysroot), so it doesn't work.
> 
> It seems the intended script was
> https://trac.webkit.org/attachment/wiki/JSCOnly/
> CrossBuildAndRemoteTestJSCLinux/cross-ldd.

That is correct; AFAIU the script on trac is based on the crosstool-NG one, so was just trying to give credit where credit was due. Sorry for the mixup, hope that didn't take long to track down. If any other similar issue comes up, please feel free to ask for more details.