Bug 194808 - REGRESSION (Safari 12): SafariForWebkitDevelopment won't load any pages on Mojave, nightly builds broken
Summary: REGRESSION (Safari 12): SafariForWebkitDevelopment won't load any pages on Mo...
Status: RESOLVED MOVED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebCore Misc. (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Alex Christensen
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2019-02-19 00:05 PST by Alexey Stoletny
Modified: 2022-11-14 03:32 PST (History)
18 users (show)

See Also:


Attachments
output1 (142 bytes, text/plain)
2019-02-20 08:58 PST, Alexey Stoletny
no flags Details
Output2 (864 bytes, text/plain)
2019-02-20 08:59 PST, Alexey Stoletny
no flags Details
Image showing what happens if I try to go to example.com (nothing) (904.23 KB, image/png)
2019-02-20 08:59 PST, Alexey Stoletny
no flags Details
Git log details (696 bytes, text/plain)
2019-02-20 09:00 PST, Alexey Stoletny
no flags Details
Result of Tools/Scripts/debug-safari (with lots of spammed [History] errors removed) (14.45 KB, text/plain)
2019-02-20 10:30 PST, Nik K
no flags Details
Patch (1.68 KB, patch)
2019-05-31 09:08 PDT, Alex Christensen
darin: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Alexey Stoletny 2019-02-19 00:05:45 PST
I am able to pull and build WebKit, and run it inside of iOS simulator or MiniBrowser. However, actual SafariForWebkitDevelopment doesn't run it with the following error:

SafariForWebKitDevelopment[81259:1239156] +[NSXPCSharedListener endpointForReply:withListenerName:]: an error occurred while attempting to obtain endpoint for listener 'SandboxBroker': Connection invalid

I tried removing the restricted flags, disabling SIP, unsigning it and a lot of other things, but I can't get it to work. 

Is there something I am missing?
Comment 1 Jonathan Bedard 2019-02-19 12:03:29 PST
It's going to take me a bit to get a machine matching this state, this seems like a bug.

That being said, I wonder if 'Tools/Scripts/run-safari' (launches system Safari with built WebKit in the dyld path) will give you what you need.
Comment 2 Alexey Stoletny 2019-02-19 22:03:32 PST
Thank you for your reply, Jonathan.

I am using run-safari --debug to run my debug build of WebKit, and by default it uses SafariForWebkitDevelopment, which does not run due to the sandbox issue.

I have modified webkitdirs.pm to use Safari, but in this case the actual binaries do not get picked up by Safari for some reason (attaching to it in a debugger shows that it has loaded the System version of WebKit regardless of the DYLD path).

Here are the things I have tried:

1) Removing 'restricted' flag from SafariForWebkitDevelopment
2) Allowing it 'full disk access' in Mojave System Preferences
3) Signing and re-signing with entitlements matching that of the MiniBrowser (which works)
4) 'defaults write com.apple.WebProcess DisableSandbox YES'
5) SafariTechPreview

None of this seems to have any effect.

Here's a thread where the same is discussed with no solution: https://stackoverflow.com/questions/54436381/running-safari-with-debug-version-of-webkit

I have also tried 3 different macOS machines with the same unsuccessful result.

Thank you for looking into it!
Comment 3 Jonathan Bedard 2019-02-20 08:38:12 PST
I'm not convinced that the sandbox is actually the error that's causing you problems.

Trying to reproduce with r241785 Mojave 18D109, I see the SandboxBroker error, but Safari runs just fine.

Could you include your WebKit version (use 'svn info') your exact Mojave build (sw_vers) and then pipe stdout and stderr from run-safari into a file and upload it to this bug (Tools/Scripts/run-safari 1>out.txt 2>out.txt)
Comment 4 Alexey Stoletny 2019-02-20 08:58:35 PST
Created attachment 362500 [details]
output1
Comment 5 Alexey Stoletny 2019-02-20 08:59:08 PST
Created attachment 362501 [details]
Output2
Comment 6 Alexey Stoletny 2019-02-20 08:59:52 PST
Created attachment 362502 [details]
Image showing what happens if I try to go to example.com (nothing)
Comment 7 Alexey Stoletny 2019-02-20 09:00:25 PST
Created attachment 362503 [details]
Git log details
Comment 8 Alexey Stoletny 2019-02-20 09:08:46 PST
Jonathan,

Thank you so much for looking into this!

I submitted the details above, I used git mirror (not svn), and here is the version info:

ProductName:	Mac OS X
ProductVersion:	10.14.3
BuildVersion:	18D109

I think I wasn't very accurate in my bug description, SafariForWebkitDevelopment does launch, but going to any specific URL is not possible (if I type up example.com and hit enter it wouldn't go).

From that, I assumed it was related to sandbox, however it may have been related to WindowServer connection failing (see output2).

So, when you run WebKit in SafariForWebkitDevelopment, are you able to navigate to sites and use it as normal? I've had a few colleagues try this with the same result as I had: SafariForWebkitDevelopment loads but is unusable (sorry for lack of clarity in the beginning).

My goal is to be able to launch SafariForWebkitDevelopment with my built WebKit, modify it and have an ability to debug.

Sincerely appreciate you looking into this!
Comment 9 Jonathan Bedard 2019-02-20 09:20:14 PST
Ok, so, I can reproduce that problem. The next thing I'd like to figure out is if this is a regression.

I've also noticed another problem, if you click on one of the 'Favorites' site, looks like we fail to open the LocalStorageDatabase, do you see that problem as well?
Comment 10 Alexey Stoletny 2019-02-20 09:35:31 PST
Thank you Jonathan, I have seen this but it doesn't show the LocalStorageDatabase error now.

Also, I have been trying to start SafariForWebkitDevelopment directly without run-safari with system Webkit, and it seems to have the same issue:

/Applications/Safari.app/Contents/MacOS/SafariForWebKitDevelopment

Also, for timeline info and regression, I have tried it in November 2018 with the same issue, but didn't report it then as I haven't spent enough time investigating on my own.
Comment 11 Nik K 2019-02-20 10:30:20 PST
Created attachment 362511 [details]
Result of Tools/Scripts/debug-safari (with lots of spammed [History] errors removed)

There are lots of errors, which seem to be similar to what happens when your app has sandboxing enabled, but outgoing (client) connections are prohibited.
Comment 12 Jonathan Bedard 2019-02-20 13:39:52 PST
(In reply to Alexey Stoletny from comment #10)
> Thank you Jonathan, I have seen this but it doesn't show the
> LocalStorageDatabase error now.

This may be something unique to my configuration, one of my machines has this bug, but not the other.

> ...
Comment 13 Alexey Stoletny 2019-02-20 15:11:19 PST
(In reply to Jonathan Bedard from comment #12)
> (In reply to Alexey Stoletny from comment #10)
> > Thank you Jonathan, I have seen this but it doesn't show the
> > LocalStorageDatabase error now.
> 
> This may be something unique to my configuration, one of my machines has
> this bug, but not the other.
> 
> > ...

Thank you for letting me know! So far I tried 2 clean installs with Xcode and the problem persists.
Comment 14 Jonathan Bedard 2019-02-20 17:02:55 PST
(In reply to Alexey Stoletny from comment #13)
> ...
> 
> Thank you for letting me know! So far I tried 2 clean installs with Xcode
> and the problem persists.

I doubt that will help.

Alexey did some testing today, apparently this is a problem all the way back to the first WebKit Nightlies published for Mojave...I've got a few ideas that I'll be trying, I'll keep this bug updated.
Comment 15 Alexey Stoletny 2019-02-20 17:04:36 PST
(In reply to Jonathan Bedard from comment #14)
> (In reply to Alexey Stoletny from comment #13)
> > ...
> > 
> > Thank you for letting me know! So far I tried 2 clean installs with Xcode
> > and the problem persists.
> 
> I doubt that will help.
> 
> Alexey did some testing today, apparently this is a problem all the way back
> to the first WebKit Nightlies published for Mojave...I've got a few ideas
> that I'll be trying, I'll keep this bug updated.

Thank you! Do you think using a different macOS version will help in the meanwhile?
Comment 16 Jonathan Bedard 2019-02-20 17:31:46 PST
(In reply to Alexey Stoletny from comment #15)
> (In reply to Jonathan Bedard from comment #14)
> > (In reply to Alexey Stoletny from comment #13)
> > > ...
> > > 
> > > Thank you for letting me know! So far I tried 2 clean installs with Xcode
> > > and the problem persists.
> > 
> > I doubt that will help.
> > 
> > Alexey did some testing today, apparently this is a problem all the way back
> > to the first WebKit Nightlies published for Mojave...I've got a few ideas
> > that I'll be trying, I'll keep this bug updated.
> 
> Thank you! Do you think using a different macOS version will help in the
> meanwhile?

There is a good chance that High Sierra will not have this bug, although I haven't verified that.
Comment 17 Alexey Stoletny 2019-02-20 17:35:47 PST
Thank you! My colleague tested on it, but I will give it another try and post back. Might Sierra be a better choice (if I can still get it)?
Comment 18 Alexey Stoletny 2019-02-21 03:25:41 PST
Turns out it only works on High Sierra but on Safari 11, and not Safari 12, so I have to use an older version of WebKit to make sure it is still compatible (the latest isn't, somehow).

Hope this helps isolate the issue.
Comment 19 Radar WebKit Bug Importer 2019-02-21 09:34:01 PST
<rdar://problem/48278497>
Comment 20 Jonathan Bedard 2019-02-21 17:09:37 PST
I'm still working on a permanent solution, but I figured I would update this bug since I have a pretty good work-around.

Turns out this is a problem loading some private Safari frameworks. If those frameworks are in the build directory, things should work. As a temporary work-around, you could copy everything matching /System/Library/PrivateFrameworks/Safari*.framework to your WebKit build directory (WebKit/WebKitBuild/Debug, WebKit/WebKitBuild/Release).
Comment 21 Alexey Stoletny 2019-02-21 19:55:19 PST
Thank you for letting me know!

I copied these to WebKit/WebKitBuild/Debug, but the issue remains, unfortunately. IS there anything else that should be done?

$ cp -R /System/Library/PrivateFrameworks/Safari*.framework .
Comment 22 Jonathan Bedard 2019-02-22 08:48:17 PST
(In reply to Alexey Stoletny from comment #21)
> Thank you for letting me know!
> 
> I copied these to WebKit/WebKitBuild/Debug, but the issue remains,
> unfortunately. IS there anything else that should be done?
> 
> $ cp -R /System/Library/PrivateFrameworks/Safari*.framework .

I did notice that the first load didn't work, but all subsequent ones did. I also wonder if that search missed /System/Library/PrivateFrameworks/Safari.framework.
Comment 23 Alexey Stoletny 2019-02-28 20:25:17 PST
Thank you, I tried but haven't gotten it to work so far on my Mojave.

Do you possibly have an update for a permanent fix?
Comment 24 Alexey Proskuryakov 2019-03-01 09:11:08 PST
Still investigating. This is not something that we are likely to forget about, but chances are that the fix will be outside WebKit.
Comment 25 Alexey Stoletny 2019-03-01 19:48:30 PST
Thank you for letting me know.
Comment 26 Joanmarie Diggs 2019-03-25 07:40:44 PDT
I'm seeing the problem described in the opening report as well, using Mojave.

I don't have this problem using the nightly downloadable binary.
Comment 27 Jonathan Bedard 2019-03-25 09:18:36 PDT
(In reply to Joanmarie Diggs (irc: joanie) from comment #26)
> I'm seeing the problem described in the opening report as well, using Mojave.
> 
> I don't have this problem using the nightly downloadable binary.

It surprises me that you don't have this problem with nightlies. Do you have any other information about your configuration that might be different from a fresh install? Of particular interest would be System Integrity Protection along with anything you might have changed in 'Security & Privacy' about where you trust app from.
Comment 28 Joanmarie Diggs 2019-03-25 10:42:08 PDT
(In reply to Jonathan Bedard from comment #27)
 
> It surprises me that you don't have this problem with nightlies. Do you have
> any other information about your configuration that might be different from
> a fresh install? Of particular interest would be System Integrity Protection
> along with anything you might have changed in 'Security & Privacy' about
> where you trust app from.

In terms of Security & Privacy, the only things which I *believe* I've changed are:

* No apps have access to Contacts, Calendars, Reminders, Photos, Camera, Microphone

* Under Accessibility: Dropbox, Synergy, and iTerm are enabled. (iTerm is there because I have written an Accessibility API Test Adaptor for W3C-related ARIA testing).

* Full Disk Access: iTerm is enabled. (I think that wound up being necessary for the aforementioned ATTA, but I don't recall for sure now.)

* Analytics: Everything is disabled.

Regarding System Integrity Protection: That's ringing a (very, very vague) bell. Would you happen to know if there's a quick way to check whether or not I've changed something there -- and if so, what it might be?

And thanks for your help with this!!
Comment 29 Alexey Proskuryakov 2019-03-25 10:47:40 PDT
> Would you happen to know if there's a quick way to check whether or not I've changed something there

csrutil status
Comment 30 Joanmarie Diggs 2019-03-25 11:00:06 PDT
(In reply to Alexey Proskuryakov from comment #29)
> > Would you happen to know if there's a quick way to check whether or not I've changed something there
> 
> csrutil status

System Integrity Protection status: enabled.
Comment 31 Alexey Proskuryakov 2019-03-25 11:05:29 PDT
I guess one more questions that needs to be asked: which versions of macOS, Safari and which WebKit archive work for you? Also, what exactly were the steps that worked?
Comment 32 Joanmarie Diggs 2019-03-25 12:47:18 PDT
(In reply to Alexey Proskuryakov from comment #31)
> I guess one more questions that needs to be asked: which versions of macOS,
> Safari and which WebKit archive work for you? Also, what exactly were the
> steps that worked?

Steps to reproduce:
1. Launch Safari
2. In the address/location bar, type apple.com* and press Return

*Any site will do, however.

~~~~

I'm using Mojave version 10.14.3.

What works:
* System version: v12.0.3 (14606.4.5)
* Safari version: v12.2   (14608.1.9.1)

The above work when:
* Launched via Dock
* Launched via iTerm

What doesn't work: WebKit built locally
 * Launched in iTerm via Tools/Scripts/run-safari
 * Launched in iTerm via Tools/Scripts/run-safari --no-saved-state
 * Launched in iTerm via Tools/Scripts/run-minibrowser

run-minibrowser reliably gives me an endless error spew (WebContent process crashed; reloading) until I terminate the process. No window appears.

run-safari does give me a window and looks normal, but loading sites via address/location bar, or command line, or by clicking favorites fails.
Comment 33 Alex Christensen 2019-05-31 09:08:52 PDT
Created attachment 371061 [details]
Patch
Comment 34 Alex Christensen 2019-05-31 09:19:10 PDT
Could Alexey or Joanmarie please verify that downloading the latest nightly and making this change to the script makes it able to open pages?  Try clicking on the favorites icons, too.  It fixed the issue for my clean install of Mojave.
Comment 35 Alex Christensen 2019-05-31 09:20:22 PDT
http://trac.webkit.org/r245965
Comment 36 Jonathan Bedard 2019-05-31 10:14:31 PDT
Touched base with Alex about this just now.

His change fixes the problem where Safari won't launch, but the search bar is still broken. Keeping this bug open to track that problem, although there is a good chance it's actually a Safari bug.
Comment 37 Carlos Bentzen 2019-07-15 11:39:50 PDT
I'm hitting this while developing Picture-in-Picture API (bug 182688).

Address bar doesn't work on SafariForWebkitDevelopment, neither is Developer Tools enabled.

Address bar and Developer Tools works on MiniBrowser but then PIP isn't enabled there. My guess is the pip window creation works only in Safari but haven't dig deep enough yet.

With all that said, the result is I can't test easily webpages on Safari while developing. Currently opening samples via Google search but can't inspect them :/

Tried the workaround described in this thread but didn't work for me. Any update on another workaround or a definite fix?

Thanks!
Comment 38 Joanmarie Diggs 2019-08-02 12:25:35 PDT
Sorry, I missed the ping.

As was already pointed out, the search bar is still broken for Safari. For me, the minibrowser search bar works. And I *think* that is good enough (tm) for my personal needs. Thanks!
Comment 39 Alexey Proskuryakov 2019-08-14 08:47:43 PDT
Does this still reproduce with the latest macOS Catalina beta for you?
Comment 40 Carlos Bentzen 2019-08-14 12:05:38 PDT
(In reply to Alexey Proskuryakov from comment #39)
> Does this still reproduce with the latest macOS Catalina beta for you?

I tested it two weeks ago and I couldn't event get to run with `run-safari` script. Something related to the application needing to be signed, but I don't remember exactly now. At that time, in Mojave, the address bar and Web inspector were still broken.

I needed to downgrade to High Sierra to get SafariForWebkitDevelopment to work :/ So I can't report back if it's working on Catalina or Mojave currently.
Comment 41 Carlos Bentzen 2019-10-09 08:45:51 PDT
Can anybody on macOS Catalina tell me if this bug happens in Safari 13?

I'm using macOS High Sierra currently, due to this bug. However, if it works in Catalina, then I will upgrade to it.
Comment 42 Jonathan Bedard 2019-10-09 10:18:54 PDT
(In reply to Carlos Eduardo Ramalho from comment #41)
> Can anybody on macOS Catalina tell me if this bug happens in Safari 13?
> 
> I'm using macOS High Sierra currently, due to this bug. However, if it works
> in Catalina, then I will upgrade to it.

I have a GM Catalina machine I built with this morning and run-safari works.

The issue you ran into 2 weeks ago was almost certainly https://bugs.webkit.org/show_bug.cgi?id=201616
Comment 43 Carlos Bentzen 2019-10-09 12:00:18 PDT
(In reply to Jonathan Bedard from comment #42)
> (In reply to Carlos Eduardo Ramalho from comment #41)
> > Can anybody on macOS Catalina tell me if this bug happens in Safari 13?
> > 
> > I'm using macOS High Sierra currently, due to this bug. However, if it works
> > in Catalina, then I will upgrade to it.
> 
> I have a GM Catalina machine I built with this morning and run-safari works.
> 
> The issue you ran into 2 weeks ago was almost certainly
> https://bugs.webkit.org/show_bug.cgi?id=201616

Thanks Jonathan! I'll update my system :)
Comment 44 Ferguson 2020-06-14 05:25:09 PDT
I faced with the same issue in Catalina latest version: 10.15.5 (19F101).
'''
2020-06-14 15:20:45.221 SafariForWebKitDevelopment[11197:99781] +[NSXPCSharedListener endpointForReply:withListenerName:]: an error occurred while attempting to obtain endpoint for listener 'SandboxBroker': Connection invalid
'''

When i try to run-safari with arg of url or local path it doesn't work. Still stopped in empty screen with empty url-bar, in this state You can't navigate or load any url.
Comment 45 Ferguson 2020-07-22 04:56:35 PDT
Some one can help with this issue?
Comment 46 Brent Fulgham 2022-02-10 20:43:34 PST
The fix for this issue was needed outside the WebKit project, therefore this is being resolved as 'Moved'.