Bug 194808 – REGRESSION (Safari 12): SafariForWebkitDevelopment won't load any pages on Mojave, nightly builds broken

• Status: RESOLVED
• Resolution: MOVED
• Component: WebCore Misc.

[Alexey Stoletny]

I am able to pull and build WebKit, and run it inside of iOS simulator or MiniBrowser. However, actual SafariForWebkitDevelopment doesn't run it with the following error:

SafariForWebKitDevelopment[81259:1239156] +[NSXPCSharedListener endpointForReply:withListenerName:]: an error occurred while attempting to obtain endpoint for listener 'SandboxBroker': Connection invalid

I tried removing the restricted flags, disabling SIP, unsigning it and a lot of other things, but I can't get it to work. 

Is there something I am missing?

[Jonathan Bedard]

It's going to take me a bit to get a machine matching this state, this seems like a bug.

That being said, I wonder if 'Tools/Scripts/run-safari' (launches system Safari with built WebKit in the dyld path) will give you what you need.

[Alexey Stoletny]

Thank you for your reply, Jonathan.

I am using run-safari --debug to run my debug build of WebKit, and by default it uses SafariForWebkitDevelopment, which does not run due to the sandbox issue.

I have modified webkitdirs.pm to use Safari, but in this case the actual binaries do not get picked up by Safari for some reason (attaching to it in a debugger shows that it has loaded the System version of WebKit regardless of the DYLD path).

Here are the things I have tried:

1) Removing 'restricted' flag from SafariForWebkitDevelopment
2) Allowing it 'full disk access' in Mojave System Preferences
3) Signing and re-signing with entitlements matching that of the MiniBrowser (which works)
4) 'defaults write com.apple.WebProcess DisableSandbox YES'
5) SafariTechPreview

None of this seems to have any effect.

Here's a thread where the same is discussed with no solution: https://stackoverflow.com/questions/54436381/running-safari-with-debug-version-of-webkit

I have also tried 3 different macOS machines with the same unsuccessful result.

Thank you for looking into it!

[Jonathan Bedard]

I'm not convinced that the sandbox is actually the error that's causing you problems.

Trying to reproduce with r241785 Mojave 18D109, I see the SandboxBroker error, but Safari runs just fine.

Could you include your WebKit version (use 'svn info') your exact Mojave build (sw_vers) and then pipe stdout and stderr from run-safari into a file and upload it to this bug (Tools/Scripts/run-safari 1>out.txt 2>out.txt)

[Alexey Stoletny]

Created attachment 362500 [details]
output1

[Alexey Stoletny]

Created attachment 362501 [details]
Output2

[Alexey Stoletny]

Created attachment 362502 [details]
Image showing what happens if I try to go to example.com (nothing)

[Alexey Stoletny]

Created attachment 362503 [details]
Git log details

[Alexey Stoletny]

Jonathan,

Thank you so much for looking into this!

I submitted the details above, I used git mirror (not svn), and here is the version info:

ProductName:	Mac OS X
ProductVersion:	10.14.3
BuildVersion:	18D109

I think I wasn't very accurate in my bug description, SafariForWebkitDevelopment does launch, but going to any specific URL is not possible (if I type up example.com and hit enter it wouldn't go).

From that, I assumed it was related to sandbox, however it may have been related to WindowServer connection failing (see output2).

So, when you run WebKit in SafariForWebkitDevelopment, are you able to navigate to sites and use it as normal? I've had a few colleagues try this with the same result as I had: SafariForWebkitDevelopment loads but is unusable (sorry for lack of clarity in the beginning).

My goal is to be able to launch SafariForWebkitDevelopment with my built WebKit, modify it and have an ability to debug.

Sincerely appreciate you looking into this!

[Jonathan Bedard]

Ok, so, I can reproduce that problem. The next thing I'd like to figure out is if this is a regression.

I've also noticed another problem, if you click on one of the 'Favorites' site, looks like we fail to open the LocalStorageDatabase, do you see that problem as well?

[Alexey Stoletny]

Thank you Jonathan, I have seen this but it doesn't show the LocalStorageDatabase error now.

Also, I have been trying to start SafariForWebkitDevelopment directly without run-safari with system Webkit, and it seems to have the same issue:

/Applications/Safari.app/Contents/MacOS/SafariForWebKitDevelopment

Also, for timeline info and regression, I have tried it in November 2018 with the same issue, but didn't report it then as I haven't spent enough time investigating on my own.

[Nik K]

Created attachment 362511 [details]
Result of Tools/Scripts/debug-safari (with lots of spammed [History] errors removed)

There are lots of errors, which seem to be similar to what happens when your app has sandboxing enabled, but outgoing (client) connections are prohibited.

[Jonathan Bedard]

(In reply to Alexey Stoletny from comment #10)
> Thank you Jonathan, I have seen this but it doesn't show the
> LocalStorageDatabase error now.

This may be something unique to my configuration, one of my machines has this bug, but not the other.

> ...

[Alexey Stoletny]

(In reply to Jonathan Bedard from comment #12)
> (In reply to Alexey Stoletny from comment #10)
> > Thank you Jonathan, I have seen this but it doesn't show the
> > LocalStorageDatabase error now.
> 
> This may be something unique to my configuration, one of my machines has
> this bug, but not the other.
> 
> > ...

Thank you for letting me know! So far I tried 2 clean installs with Xcode and the problem persists.

[Jonathan Bedard]

(In reply to Alexey Stoletny from comment #13)
> ...
> 
> Thank you for letting me know! So far I tried 2 clean installs with Xcode
> and the problem persists.

I doubt that will help.

Alexey did some testing today, apparently this is a problem all the way back to the first WebKit Nightlies published for Mojave...I've got a few ideas that I'll be trying, I'll keep this bug updated.

[Alexey Stoletny]

(In reply to Jonathan Bedard from comment #14)
> (In reply to Alexey Stoletny from comment #13)
> > ...
> > 
> > Thank you for letting me know! So far I tried 2 clean installs with Xcode
> > and the problem persists.
> 
> I doubt that will help.
> 
> Alexey did some testing today, apparently this is a problem all the way back
> to the first WebKit Nightlies published for Mojave...I've got a few ideas
> that I'll be trying, I'll keep this bug updated.

Thank you! Do you think using a different macOS version will help in the meanwhile?

[Jonathan Bedard]

(In reply to Alexey Stoletny from comment #15)
> (In reply to Jonathan Bedard from comment #14)
> > (In reply to Alexey Stoletny from comment #13)
> > > ...
> > > 
> > > Thank you for letting me know! So far I tried 2 clean installs with Xcode
> > > and the problem persists.
> > 
> > I doubt that will help.
> > 
> > Alexey did some testing today, apparently this is a problem all the way back
> > to the first WebKit Nightlies published for Mojave...I've got a few ideas
> > that I'll be trying, I'll keep this bug updated.
> 
> Thank you! Do you think using a different macOS version will help in the
> meanwhile?

There is a good chance that High Sierra will not have this bug, although I haven't verified that.

[Alexey Stoletny]

Thank you! My colleague tested on it, but I will give it another try and post back. Might Sierra be a better choice (if I can still get it)?

[Alexey Stoletny]

Turns out it only works on High Sierra but on Safari 11, and not Safari 12, so I have to use an older version of WebKit to make sure it is still compatible (the latest isn't, somehow).

Hope this helps isolate the issue.

[Radar WebKit Bug Importer]

<rdar://problem/48278497>

[Jonathan Bedard]

I'm still working on a permanent solution, but I figured I would update this bug since I have a pretty good work-around.

Turns out this is a problem loading some private Safari frameworks. If those frameworks are in the build directory, things should work. As a temporary work-around, you could copy everything matching /System/Library/PrivateFrameworks/Safari*.framework to your WebKit build directory (WebKit/WebKitBuild/Debug, WebKit/WebKitBuild/Release).

[Alexey Stoletny]

Thank you for letting me know!

I copied these to WebKit/WebKitBuild/Debug, but the issue remains, unfortunately. IS there anything else that should be done?

$ cp -R /System/Library/PrivateFrameworks/Safari*.framework .

[Jonathan Bedard]

(In reply to Alexey Stoletny from comment #21)
> Thank you for letting me know!
> 
> I copied these to WebKit/WebKitBuild/Debug, but the issue remains,
> unfortunately. IS there anything else that should be done?
> 
> $ cp -R /System/Library/PrivateFrameworks/Safari*.framework .

I did notice that the first load didn't work, but all subsequent ones did. I also wonder if that search missed /System/Library/PrivateFrameworks/Safari.framework.

[Alexey Stoletny]

Thank you, I tried but haven't gotten it to work so far on my Mojave.

Do you possibly have an update for a permanent fix?

[Alexey Proskuryakov]

Still investigating. This is not something that we are likely to forget about, but chances are that the fix will be outside WebKit.

[Alexey Stoletny]

Thank you for letting me know.

[Joanmarie Diggs]

I'm seeing the problem described in the opening report as well, using Mojave.

I don't have this problem using the nightly downloadable binary.

[Jonathan Bedard]

(In reply to Joanmarie Diggs (irc: joanie) from comment #26)
> I'm seeing the problem described in the opening report as well, using Mojave.
> 
> I don't have this problem using the nightly downloadable binary.

It surprises me that you don't have this problem with nightlies. Do you have any other information about your configuration that might be different from a fresh install? Of particular interest would be System Integrity Protection along with anything you might have changed in 'Security & Privacy' about where you trust app from.

[Joanmarie Diggs]

(In reply to Jonathan Bedard from comment #27)
 
> It surprises me that you don't have this problem with nightlies. Do you have
> any other information about your configuration that might be different from
> a fresh install? Of particular interest would be System Integrity Protection
> along with anything you might have changed in 'Security & Privacy' about
> where you trust app from.

In terms of Security & Privacy, the only things which I *believe* I've changed are:

* No apps have access to Contacts, Calendars, Reminders, Photos, Camera, Microphone

* Under Accessibility: Dropbox, Synergy, and iTerm are enabled. (iTerm is there because I have written an Accessibility API Test Adaptor for W3C-related ARIA testing).

* Full Disk Access: iTerm is enabled. (I think that wound up being necessary for the aforementioned ATTA, but I don't recall for sure now.)

* Analytics: Everything is disabled.

Regarding System Integrity Protection: That's ringing a (very, very vague) bell. Would you happen to know if there's a quick way to check whether or not I've changed something there -- and if so, what it might be?

And thanks for your help with this!!

[Alexey Proskuryakov]

> Would you happen to know if there's a quick way to check whether or not I've changed something there

csrutil status

[Joanmarie Diggs]

(In reply to Alexey Proskuryakov from comment #29)
> > Would you happen to know if there's a quick way to check whether or not I've changed something there
> 
> csrutil status

System Integrity Protection status: enabled.

[Alexey Proskuryakov]

I guess one more questions that needs to be asked: which versions of macOS, Safari and which WebKit archive work for you? Also, what exactly were the steps that worked?

[Joanmarie Diggs]

(In reply to Alexey Proskuryakov from comment #31)
> I guess one more questions that needs to be asked: which versions of macOS,
> Safari and which WebKit archive work for you? Also, what exactly were the
> steps that worked?

Steps to reproduce:
1. Launch Safari
2. In the address/location bar, type apple.com* and press Return

*Any site will do, however.

~~~~

I'm using Mojave version 10.14.3.

What works:
* System version: v12.0.3 (14606.4.5)
* Safari version: v12.2   (14608.1.9.1)

The above work when:
* Launched via Dock
* Launched via iTerm

What doesn't work: WebKit built locally
 * Launched in iTerm via Tools/Scripts/run-safari
 * Launched in iTerm via Tools/Scripts/run-safari --no-saved-state
 * Launched in iTerm via Tools/Scripts/run-minibrowser

run-minibrowser reliably gives me an endless error spew (WebContent process crashed; reloading) until I terminate the process. No window appears.

run-safari does give me a window and looks normal, but loading sites via address/location bar, or command line, or by clicking favorites fails.

[Alex Christensen]

Created attachment 371061 [details]
Patch

[Alex Christensen]

Could Alexey or Joanmarie please verify that downloading the latest nightly and making this change to the script makes it able to open pages?  Try clicking on the favorites icons, too.  It fixed the issue for my clean install of Mojave.

[Alex Christensen]

http://trac.webkit.org/r245965

[Jonathan Bedard]

Touched base with Alex about this just now.

His change fixes the problem where Safari won't launch, but the search bar is still broken. Keeping this bug open to track that problem, although there is a good chance it's actually a Safari bug.

[Carlos Bentzen]

I'm hitting this while developing Picture-in-Picture API (bug 182688).

Address bar doesn't work on SafariForWebkitDevelopment, neither is Developer Tools enabled.

Address bar and Developer Tools works on MiniBrowser but then PIP isn't enabled there. My guess is the pip window creation works only in Safari but haven't dig deep enough yet.

With all that said, the result is I can't test easily webpages on Safari while developing. Currently opening samples via Google search but can't inspect them :/

Tried the workaround described in this thread but didn't work for me. Any update on another workaround or a definite fix?

Thanks!

[Joanmarie Diggs]

Sorry, I missed the ping.

As was already pointed out, the search bar is still broken for Safari. For me, the minibrowser search bar works. And I *think* that is good enough (tm) for my personal needs. Thanks!

[Alexey Proskuryakov]

Does this still reproduce with the latest macOS Catalina beta for you?

[Carlos Bentzen]

(In reply to Alexey Proskuryakov from comment #39)
> Does this still reproduce with the latest macOS Catalina beta for you?

I tested it two weeks ago and I couldn't event get to run with `run-safari` script. Something related to the application needing to be signed, but I don't remember exactly now. At that time, in Mojave, the address bar and Web inspector were still broken.

I needed to downgrade to High Sierra to get SafariForWebkitDevelopment to work :/ So I can't report back if it's working on Catalina or Mojave currently.

[Carlos Bentzen]

Can anybody on macOS Catalina tell me if this bug happens in Safari 13?

I'm using macOS High Sierra currently, due to this bug. However, if it works in Catalina, then I will upgrade to it.

[Jonathan Bedard]

(In reply to Carlos Eduardo Ramalho from comment #41)
> Can anybody on macOS Catalina tell me if this bug happens in Safari 13?
> 
> I'm using macOS High Sierra currently, due to this bug. However, if it works
> in Catalina, then I will upgrade to it.

I have a GM Catalina machine I built with this morning and run-safari works.

The issue you ran into 2 weeks ago was almost certainly https://bugs.webkit.org/show_bug.cgi?id=201616

[Carlos Bentzen]

(In reply to Jonathan Bedard from comment #42)
> (In reply to Carlos Eduardo Ramalho from comment #41)
> > Can anybody on macOS Catalina tell me if this bug happens in Safari 13?
> > 
> > I'm using macOS High Sierra currently, due to this bug. However, if it works
> > in Catalina, then I will upgrade to it.
> 
> I have a GM Catalina machine I built with this morning and run-safari works.
> 
> The issue you ran into 2 weeks ago was almost certainly
> https://bugs.webkit.org/show_bug.cgi?id=201616

Thanks Jonathan! I'll update my system :)

[Ferguson]

I faced with the same issue in Catalina latest version: 10.15.5 (19F101).
'''
2020-06-14 15:20:45.221 SafariForWebKitDevelopment[11197:99781] +[NSXPCSharedListener endpointForReply:withListenerName:]: an error occurred while attempting to obtain endpoint for listener 'SandboxBroker': Connection invalid
'''

When i try to run-safari with arg of url or local path it doesn't work. Still stopped in empty screen with empty url-bar, in this state You can't navigate or load any url.

[Ferguson]

Some one can help with this issue?

[Brent Fulgham]

The fix for this issue was needed outside the WebKit project, therefore this is being resolved as 'Moved'.