Bug 236829 – Can't run WebKit (local build & nightly build) with Safari on Apple Silicon

• Status: RESOLVED
• Resolution: CONFIGURATION CHANGED
• Component: WebKit Misc.

[Lingyun Cai]

On my M1 Max MacBook Pro, I tried to build and run Webkit from source using the scripts tool:
$ build-webkit --debug
$ run-safari --debug

Safari was launched, but it used the frameworks installed in /System/Library/Frameworks rather than from my build directory.
- My logs and breakpoints were not working.
- I can't see an additional version info '613+' shown in 'About Safari' version row.
- I can see from Activity Monitor that the cwd for 'SafariForWebkitDevelopment Web Content' process was '/System/Library/Frameworks/Webkit.Framework/....'.

I also tried to build using Xcode and run with Safari.app, it still didn't load the frameworks/libraries from my build directory. However, built and ran with MiniBrowser.app worked.

Besides, I tried the latest Webkit Build Archive (r290089) and ran with run-webkit-archive script, it also didn't load the frameworks/libraries in the specified 'Release' directory as well. The log from run-webkit-archive script was shown as below, but I don't think this 'SandboxBroker' related error could be the root cause as on my MBP M1 the same log shown while Webkit Build Archive was running successfully with Safari.

Setting DYLD FRAMEWORK and LIBRARY paths to /Users/xxx/Downloads/290089/Release
2022-02-18 15:49:01.062 SafariForWebKitDevelopment[90714:301002] +[NSXPCSharedListener endpointForReply:withListenerName:]: an error occurred while attempting to obtain endpoint for listener 'SandboxBroker': Connection invalid
2022-02-18 15:49:01.111 SafariForWebKitDevelopment[90714:301021] +[NSXPCSharedListener endpointForReply:withListenerName:]: an error occurred while attempting to obtain endpoint for listener 'SandboxBroker': Connection invalid
2022-02-18 15:49:01.122 SafariForWebKitDevelopment[90714:301044] +[NSXPCSharedListener endpointForReply:withListenerName:]: an error occurred while attempting to obtain endpoint for listener 'SandboxBroker': Connection invalid

BTW, I've enabled 'full disk access' in System Preferences for SafariForWebkitDevelopment/Safari/Terminal/Xcode but seems to have no effect.

Below are the device configurations I'm using:
MBP M1 Max
Hardware
- Model: Macbook Pro18, 2
- Chip: Apple M1 Max
- System Firmware Version: 7429.81.3

Software
- System Version: 12.2.1 (21D62)
- Kernel Version: Darwin 21.3.0
- Safari: 15.3
- Xcode: 13.1 (13A1030d)


MBP M1
Hardware
- Model: Macbook Pro17, 1
- Chip: Apple M1
- System Firmware Version: 7429.61.2

Software
- System Version: 12.1 (21C52)
- Kernel Version: Darwin 21.2.0
- Safari: 15.1

Note that when MBP M1 Max was with the same SW versions as M1, it still failed to run Webkit with Safari. Not sure if there was anything I missed in configuring the system.

[Lingyun Cai]

Seems that env DYLD_FRAMEWORK_PATH and DYLD_LIBRARY_PATH were not working properly to dynamically load the frameworks/libraries in specific directory.

I've tried to disable System Integrity Protection on my MBP M1 Max but it still didn't help.

[Sam Sneddon [:gsnedders]]

I guess we've just changed something such that the shipping Safari is no longer compatible with WebKit ToT, which… is something that happens often.

Does MiniBrowser work for whatever needs you have, because that's a much more reliable option?

[Radar WebKit Bug Importer]

<rdar://problem/89231031>

[Alexey Proskuryakov]

I think that we'd be getting crashes if it's just an incompatible change. This seems like a different kind of issue.

Could you please check what frameworks SafariForWebkitDevelopment uses? Checking open files for it using Activity Monitor is the easiest way to do it.

Reproducing on command line directly with logging could provide some useful insight:

cd <your build directory>

DYLD_FRAMEWORK_PATH=$PWD DYLD_LIBRARY_PATH=$PWD __XPC_DYLD_FRAMEWORK_PATH=$PWD __XPC_DYLD_LIBRARY_PATH=$PWD WEBKIT_UNSET_DYLD_FRAMEWORK_PATH=YES DYLD_PRINT_LIBRARIES=YES DYLD_PRINT_SEARCHING=YES /Library/Application\ Support/Apple/Safari/SafariForWebKitDevelopment

[Alexey Proskuryakov]

Actually, it doesn't launch form /Library - need to use the /Applications/Safari.app/Contents/MacOS/SafariForWebKitDevelopment path.

I installed an appropriate version of macOS and can reproduce, so I can look into it.

[Alexey Proskuryakov]

Filed rdar://89271003 for Apple investigation. As a workaround, please use MiniBrowser for now.

[Lingyun Cai]

Thanks for the update.

I've tried to execute run-webkit-archive script in Webkit Build Archive r290089 with below env set, and it succeeded on M1 MBP but failed on M1 Max MBP to load the frameworks/libraries in this package.
DYLD_FRAMEWORK_PATH=$PWD DYLD_LIBRARY_PATH=$PWD __XPC_DYLD_FRAMEWORK_PATH=$PWD __XPC_DYLD_LIBRARY_PATH=$PWD WEBKIT_UNSET_DYLD_FRAMEWORK_PATH=YES DYLD_PRINT_LIBRARIES=YES DYLD_PRINT_SEARCHING=YES 

It might be related with the error "fat file, but missing compatible architecture (have 'x86_64,arm64', need 'arm64e')", but I'm not sure about these. Details as below.

M1 - succeeded
Log (take JavaScriptCore.framework for example, loading framework from the package, no 'fat file' error):
...
dyld[8686]: find path "/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore"
dyld[8686]:   possible path(DYLD_FRAMEWORK/LIBRARY_PATH): "/Users/lingyun/Downloads/290089/Release/JavaScriptCore.framework/Versions/A/JavaScriptCore"
dyld[8686]:   found: dylib-from-disk-to-override-cache: "/Users/lingyun/Downloads/290089/Release/JavaScriptCore.framework/Versions/A/JavaScriptCore"
dyld[8686]: <A791C970-A223-358C-9660-39E0E28C5F65> /Users/lingyun/Downloads/290089/Release/JavaScriptCore.framework/Versions/A/JavaScriptCore
...

Activity monitor (partial, only shows the directory related with frameworks/libraries):
cwd
/Users/lingyun/Downloads/290089/Release/com.apple.WebKit.WebContent.xpc
txt
/Users/lingyun/Downloads/290089/Release/com.apple.WebKit.WebContent.xpc/Contents/MacOS/com.apple.WebKit.WebContent.Development
txt
/Users/lingyun/Downloads/290089/Release/WebInspectorUI.framework/Versions/A/WebInspectorUI
txt
/usr/lib/dyld
txt
/Users/lingyun/Downloads/290089/Release/libANGLE-shared.dylib
...
txt
/Users/lingyun/Downloads/290089/Release/WebKitLegacy.framework/Versions/A/WebKitLegacy
...
txt
/Users/lingyun/Downloads/290089/Release/libwebrtc.dylib
txt
/System/Library/Fonts/SFNS.ttf
txt
/Users/lingyun/Downloads/290089/Release/WebKit.framework/Versions/A/WebKit
txt
/Users/lingyun/Downloads/290089/Release/JavaScriptCore.framework/Versions/A/JavaScriptCore
...
txt
/Users/lingyun/Downloads/290089/Release/WebCore.framework/Versions/A/WebCore

-------------------------------------------------------------------------------

M1 Max - failed
Log (loading framework from system default dir, 'fat file' error occurred):
...
dyld[1499]: find path "/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore"
dyld[1499]:   possible path(DYLD_FRAMEWORK/LIBRARY_PATH): "/Users/lingyun/Downloads/290089/Release/JavaScriptCore.framework/Versions/A/JavaScriptCore"
dyld[1499]:   found: dylib-from-disk-to-override-cache: "/Users/lingyun/Downloads/290089/Release/JavaScriptCore.framework/Versions/A/JavaScriptCore"
dyld[1499]:   found: dylib-from-disk-to-override-cache-error: "/Users/lingyun/Downloads/290089/Release/JavaScriptCore.framework/Versions/A/JavaScriptCore" => "fat file, but missing compatible architecture (have 'x86_64,arm64', need 'arm64e')"
dyld[1499]:   possible path(original path): "/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore"
dyld[1499]:   found: dylib-from-cache: (0x01F1) "/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore"
dyld[1499]: <258C04CC-8C23-3C5D-BC2D-5D64DBA3C81C> /System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore
...

Activity monitor (partial, only shows the directory related with frameworks/libraries):
cwd
/System/Library/Frameworks/WebKit.framework/Versions/A/XPCServices/com.apple.WebKit.WebContent.xpc
txt
/System/Library/Frameworks/WebKit.framework/Versions/A/XPCServices/com.apple.WebKit.WebContent.xpc/Contents/MacOS/com.apple.WebKit.WebContent
...

[Alexey Proskuryakov]

> It might be related with the error "fat file, but missing compatible architecture (have 'x86_64,arm64', need 'arm64e')", but I'm not sure about these. Details as below.

Yes, it certainly is.

I see two issues:
1. DYLD_ variables are ignored when System Integrity Protection is enabled. Disabling SIP shouldn't be required to use SafariForWebKitDevelopment in principle, but it is at this time.
2. Frameworks aren't getting used because of arm64/arm64e mismatch.

I don't understand why #2 is not happening on one of your machines.

[Lingyun Cai]

> I see two issues:
> 1. DYLD_ variables are ignored when System Integrity Protection is enabled.
> Disabling SIP shouldn't be required to use SafariForWebKitDevelopment in
> principle, but it is at this time.
> 2. Frameworks aren't getting used because of arm64/arm64e mismatch.
> 
> I don't understand why #2 is not happening on one of your machines.

I've tried below two ways of running WebKit Build Archive on my machines (SIP was disabled for both).
- On my M1 MBP, #1 succeeded but #2 failed with "arm64/arm64e mismatch"
- On my M1 Max MBP, both #1 and #2 failed with "arm64/arm64e mismatch"

#1 - set the env in terminal and execute run-webkit-archive
cd /Users/lingyun/Downloads/290089/Release
export DYLD_FRAMEWORK_PATH=$PWD
export DYLD_LIBRARY_PATH=$PWD
export __XPC_DYLD_FRAMEWORK_PATH=$PWD
export __XPC_DYLD_LIBRARY_PATH=$PWD
export WEBKIT_UNSET_DYLD_FRAMEWORK_PATH=YES
export DYLD_PRINT_LIBRARIES=YES
export DYLD_PRINT_SEARCHING=YES
cd ..
python run-webkit-archive

#2 - invoke SafariForWebKitDevelopment directly
cd /Users/lingyun/Downloads/290089/Release
DYLD_FRAMEWORK_PATH=$PWD DYLD_LIBRARY_PATH=$PWD __XPC_DYLD_FRAMEWORK_PATH=$PWD __XPC_DYLD_LIBRARY_PATH=$PWD WEBKIT_UNSET_DYLD_FRAMEWORK_PATH=YES DYLD_PRINT_LIBRARIES=YES DYLD_PRINT_SEARCHING=YES /Applications/Safari.app/Contents/MacOS/SafariForWebKitDevelopment

Please let me know if there is anything I can do on my machines to help.

[Alexey Proskuryakov]

What do "which python" and "file `which python`" say on both? I wonder if different architectures of the python binary affect SafariForWebKitDevelpoment - although I couldn't reproduce any difference by forcing it with "arch" manually.

[Lingyun Cai]

> What do "which python" and "file `which python`" say on both? I wonder if
> different architectures of the python binary affect
> SafariForWebKitDevelpoment - although I couldn't reproduce any difference by
> forcing it with "arch" manually.

Here comes the difference. My M1 Max machine has Mach-O universal binary with 2 architectures while there is only x86_64 arch python binary on my M1. If I run the run-webkit-archive script with arch command, it succeeded in loading the frameworks in my working directory.
$ arch -x86_64 python run-webkit-archive

If forcing with arm64e as below, it failed in loading the frameworks.
$ arch -arm64e python run-webkit-archive

On my M1 machine:
$ python --version
Python 2.7.18
$ which python
/usr/local/bin/python
$ file `which python`
/usr/local/bin/python: Mach-O 64-bit executable x86_64

On my M1 Max machine:
$ python --version
Python 2.7.18
$ which python
/usr/bin/python
$ file `which python`
/usr/bin/python: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]
/usr/bin/python (for architecture x86_64):	Mach-O 64-bit executable x86_64
/usr/bin/python (for architecture arm64e):	Mach-O 64-bit executable arm64e

[Alexey Proskuryakov]

Indeed, I made a mistake in my testing yesterday. It works like this too:

arch --arch x86_64 -e DYLD_FRAMEWORK_PATH=$PWD -e DYLD_LIBRARY_PATH=$PWD -e __XPC_DYLD_FRAMEWORK_PATH=$PWD -e __XPC_DYLD_LIBRARY_PATH=$PWD /Applications/Safari.app/Contents/MacOS/SafariForWebKitDevelopment

[Lingyun Cai]

Thanks for the update!

I further checked the logs of run-safari script, seems that the failure was also caused by the arm64/arm64e architecture mismatch on below 7 frameworks/libraries:
- JavaScriptCore.framework
- WebKit.framework
- WebCore.framekwork
- WebKitLegacy.framework
- WebInspectorUI.framework
- libwebrtc.dylib
- libANGLE-shared.dylib

dyld[7213]: find path "/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore"
dyld[7213]:   possible path(DYLD_FRAMEWORK/LIBRARY_PATH): "/Users/lingyun/webkit-codebase/webkit/WebKitBuild/Debug/JavaScriptCore.framework/Versions/A/JavaScriptCore"
dyld[7213]:   found: dylib-from-disk-to-override-cache: "/Users/lingyun/webkit-codebase/webkit/WebKitBuild/Debug/JavaScriptCore.framework/Versions/A/JavaScriptCore"
dyld[7213]:   found: dylib-from-disk-to-override-cache-error: "/Users/lingyun/webkit-codebase/webkit/WebKitBuild/Debug/JavaScriptCore.framework/Versions/A/JavaScriptCore" => "mach-o file, but is an incompatible architecture (have 'arm64', need 'arm64e')"
dyld[7213]:   possible path(original path): "/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore"

As the required arch is arm64e rather than arm64 for these frameworks/libraries on Apple silicon MacOS, have you considered adding arm64e to the default building architecture target of WebKit? In this case, there should be no issue in running both local build and nightly build WebKit with Safari.

[Alexey Proskuryakov]

SafariForWebKitDevelopment on macOS Ventura is arm64+x86_64, resolving this particular issue.

But it's still not working properly, using bug 247823 to track that for now.