Bug 179945 – Consider hosting pre-built `jsc` binaries somewhere official

• Status: RESOLVED
• Resolution: WORKSFORME
• Component: JavaScriptCore

[Mathias Bynens]

I imagine bots currently build and test `jsc` on every supported configuration.

It would be immensely useful if the built `jsc` binaries could be hosted somewhere official as part of the build process, if only for major releases.

This would make it easier for developers to test different JavaScriptCore versions: instead of having to build them all from source, they could just `wget` and extract a `.tar.gz` file.

[Mathias Bynens]

Side note: the JSC that’s part of stable Safari can be used directly as follows:

    /System/Library/Frameworks/JavaScriptCore.framework/Versions/A/Resources/jsc

Is there a similar way to get to the version of JSC used by the currently installed Safari Technology Preview?

[Mathias Bynens]

I noticed that it’s possible to get `jsc.exe` binaries for Windows as follows:

1. Open https://build.webkit.org/builders/Apple%20Win%20Debug%20%28Build%29
2. Click a build number, which takes you to a page like https://build.webkit.org/builders/Apple%20Win%20Debug%20%28Build%29/builds/5674
3. Click the `stdio` link below `transfer-to-s3`
4. The “S3 URL” (e.g. https://s3-us-west-2.amazonaws.com/archives.webkit.org/win-i386-debug/225094.zip) points to a ZIP file containing, among others:

    ```
    JavaScriptCore.resources (directory)
    JavaScriptCore.dll
    JavaScriptCore.pdb
    jsc.exe
    jsc.pdb
    jscLib.dll
    jscLib.pdb
    ```

Sadly, the other bots don’t seem to have a `transfer-to-s3` step. :(

[Alexey Proskuryakov]

Everything that CI builds ends up being officially available from nightly.webkit.org. I’m not sure offhand if running jsc from these archives would use the framework from the archive or the system one. It’s definitely possible to use dyld environment variables to control this, and we may already have helper scripts for that. 

None of that actually helps the use case of comparing major releases though. First, CI doesn’t build release branches, and second, retention policy may be too short for the purpose of testing old releases. 

JavaScriptCore has system dependencies, so for testing past versions, one needs a natching macOS version anyway. Nightlies are sufficient for testing trunk. So I don’t see any action that can be taken to help here. Please let us know if something is not covered yet!

[Alexey Proskuryakov]

(In reply to Mathias Bynens from comment #1)
> Side note: the JSC that’s part of stable Safari can be used directly as
> follows:
> 
>    
> /System/Library/Frameworks/JavaScriptCore.framework/Versions/A/Resources/jsc

This is the system version of the framework, which is not always the same as the one used by Safari. For example, Safari 11 on Sierra uses StagedFrameworks, not Frameworks.

[Mathias Bynens]

(In reply to Alexey Proskuryakov from comment #3)
> Everything that CI builds ends up being officially available from
> nightly.webkit.org. I’m not sure offhand if running jsc from these archives
> would use the framework from the archive or the system one. It’s definitely
> possible to use dyld environment variables to control this, and we may
> already have helper scripts for that. 
> 
> None of that actually helps the use case of comparing major releases though.
> First, CI doesn’t build release branches, and second, retention policy may
> be too short for the purpose of testing old releases. 
> 
> JavaScriptCore has system dependencies, so for testing past versions, one
> needs a natching macOS version anyway. Nightlies are sufficient for testing
> trunk. So I don’t see any action that can be taken to help here. Please let
> us know if something is not covered yet!

I know nightly.webkit.org offers such downloads for all of WebKit. I’m looking for stand-alone `jsc` binaries, however.

I’m also not interested in old jsc versions. The use case is getting a recent (as in ~head of trunk) build of `jsc`.

[Mathias Bynens]

(In reply to Mathias Bynens from comment #5)
> (In reply to Alexey Proskuryakov from comment #3)
> > Everything that CI builds ends up being officially available from
> > nightly.webkit.org. I’m not sure offhand if running jsc from these archives
> > would use the framework from the archive or the system one. It’s definitely
> > possible to use dyld environment variables to control this, and we may
> > already have helper scripts for that. 
> > 
> > None of that actually helps the use case of comparing major releases though.
> > First, CI doesn’t build release branches, and second, retention policy may
> > be too short for the purpose of testing old releases. 
> > 
> > JavaScriptCore has system dependencies, so for testing past versions, one
> > needs a natching macOS version anyway. Nightlies are sufficient for testing
> > trunk. So I don’t see any action that can be taken to help here. Please let
> > us know if something is not covered yet!
> 
> I know nightly.webkit.org offers such downloads for all of WebKit. I’m
> looking for stand-alone `jsc` binaries, however.

Oh, I see what you mean now! Those binaries can be found in `Release/jsc`. Thanks!

Is there an easy way to get similar binaries for Windows — easier than what’s described in comment 2? What about Linux?

[Mathias Bynens]

(In reply to Alexey Proskuryakov from comment #3)
> Everything that CI builds ends up being officially available from
> nightly.webkit.org. I’m not sure offhand if running jsc from these archives
> would use the framework from the archive or the system one.

Running the `jsc` binary from the latest nightly (r225145) on macOS Sierra 10.12.6 (16G1036) gives:

    $ ./jsc
    dyld: Symbol not found: __ZN3JSC19JSWebAssemblyMemory6s_infoE
      Referenced from: /Users/mathiasb/Downloads/225145/Release/./jsc (which was built for Mac OS X 10.13)
      Expected in: /System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore
     in /Users/mathiasb/Downloads/225145/Release/./jsc
    Abort trap: 6

So I guess it’s trying to use the system framework.

> It’s definitely
> possible to use dyld environment variables to control this, and we may
> already have helper scripts for that. 

Do you have any pointers here?

[Saam Barati]

(In reply to Mathias Bynens from comment #7)
> (In reply to Alexey Proskuryakov from comment #3)
> > Everything that CI builds ends up being officially available from
> > nightly.webkit.org. I’m not sure offhand if running jsc from these archives
> > would use the framework from the archive or the system one.
> 
> Running the `jsc` binary from the latest nightly (r225145) on macOS Sierra
> 10.12.6 (16G1036) gives:
> 
>     $ ./jsc
>     dyld: Symbol not found: __ZN3JSC19JSWebAssemblyMemory6s_infoE
>       Referenced from: /Users/mathiasb/Downloads/225145/Release/./jsc (which
> was built for Mac OS X 10.13)
>       Expected in:
> /System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore
>      in /Users/mathiasb/Downloads/225145/Release/./jsc
>     Abort trap: 6
> 
> So I guess it’s trying to use the system framework.
> 
> > It’s definitely
> > possible to use dyld environment variables to control this, and we may
> > already have helper scripts for that. 
> 
> Do you have any pointers here?

You will want to set
DYLD_FRAMEWORK_PATH=path/to/frameworks/directory/in/nightly

I’m not in front of a computer to get the exact path. But if you dig inside the bundle you should find a directory that has JavaScriptCore.framework.

[Mathias Bynens]

(In reply to Saam Barati from comment #8)
> (In reply to Mathias Bynens from comment #7)
> > (In reply to Alexey Proskuryakov from comment #3)
> > > Everything that CI builds ends up being officially available from
> > > nightly.webkit.org. I’m not sure offhand if running jsc from these archives
> > > would use the framework from the archive or the system one.
> > 
> > Running the `jsc` binary from the latest nightly (r225145) on macOS Sierra
> > 10.12.6 (16G1036) gives:
> > 
> >     $ ./jsc
> >     dyld: Symbol not found: __ZN3JSC19JSWebAssemblyMemory6s_infoE
> >       Referenced from: /Users/mathiasb/Downloads/225145/Release/./jsc (which
> > was built for Mac OS X 10.13)
> >       Expected in:
> > /System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore
> >      in /Users/mathiasb/Downloads/225145/Release/./jsc
> >     Abort trap: 6
> > 
> > So I guess it’s trying to use the system framework.
> > 
> > > It’s definitely
> > > possible to use dyld environment variables to control this, and we may
> > > already have helper scripts for that. 
> > 
> > Do you have any pointers here?
> 
> You will want to set
> DYLD_FRAMEWORK_PATH=path/to/frameworks/directory/in/nightly
> 
> I’m not in front of a computer to get the exact path. But if you dig inside
> the bundle you should find a directory that has JavaScriptCore.framework.

Thanks for the pointers! The `run-webkit-archive` script included in the ZIP file helped, too.

I now use a script of the following form to invoke the `jsc` binary:

    $ cat "$(which jsc)"
    #!/usr/bin/env bash
    DYLD_FRAMEWORK_PATH=/path/to/download/Release DYLD_LIBRARY_PATH=DYLD_FRAMEWORK_PATH=/path/to/download/Release /path/to/download/Release/jsc "$@"

[Mathias Bynens]

One question remains:

(In reply to Mathias Bynens from comment #6)
> Is there an easy way to get similar binaries for Windows — easier than
> what’s described in comment 2? What about Linux?

[Alexey Proskuryakov]

We can consider adding links to Windows and Linux to nightly.webkit.org. Not quite sure what all the considerations around that are, but feel free to file (separate) bugs, and maybe even ping webkit-dev mailing list explaining why this should be done.

Alternatively, what you describe in comment 2 seems feasible for a script to do.

[Mathias Bynens]

(In reply to Alexey Proskuryakov from comment #11)

> Alternatively, what you describe in comment 2 seems feasible for a script to
> do.

Yes, in fact this is what jsvu does for JSC on Windows: https://github.com/GoogleChromeLabs/jsvu I was just wondering if there was a better way.

> We can consider adding links to Windows and Linux to nightly.webkit.org. Not
> quite sure what all the considerations around that are, but feel free to
> file (separate) bugs, and maybe even ping webkit-dev mailing list explaining
> why this should be done.

https://lists.webkit.org/pipermail/webkit-dev/2017-December/029805.html