Bug 214043 – Building with the macOS Big Sur public SDK fails

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Rob Buis]

Building with the public SDK fails (Xcode 11.5 and Big Sur), there are multiple problems.

[Rob Buis]

Created attachment 403702 [details]
Patch

[Rob Buis]

Actually it seems Xcode 11.5 is not supposed to work on Big Sur, at least the command line tools refuse to install.
However the build problems with Xcode 12 are very similar. I am not very familiar with this code and how to cleanly fix it, so this is a rough patch but it unblocks me locally.
Some points:
- I do not know how Source/WTF/wtf/PlatformHave.h could test for public sdk.
- Setting _allowsHTTP3 seems problematic because it is marked readonly. How could that ever work?
- the typedef enum seems a problem for clang although it seems valid C++ to me.

Any hints appreciated! :)

[Alexey Proskuryakov]

We should make every attempt to not guard functionality on USE(APPLE_INTERNAL_SDK), because we want the open source build to be real, and to work the same as what Apple ships as much as possible.

I think that it would be easiest for someone at Apple to fix the build. Do you need it soon?

[Saagar Jha]

I'm using the Xcode 12 SDK and seeing issues as well. The first one I ran into is a missing AXSIsolatedTreeMode, which I suspect is defined in the internal SDK and hiding the build failure: https://bugs.webkit.org/show_bug.cgi?id=208691#c19. I tried working around this by patching out ENABLE(ACCESSIBILITY_ISOLATED_TREE) code and ran into this, which I fixed: https://bugs.webkit.org/show_bug.cgi?id=213911. So now I'm up to WebCore, which fails because it's missing ANGLE headers. It seems to me like the "Copy Headers" phase from the ANGLE static library target should be doing this, but it doesn't show up in the build log at all so this might be some sort of dependency tracking failure in the build system.

[Darin Adler]

Comment on attachment 403702 [details]
Patch

These changes don’t seem right. We want to require the internal SDK only if absolutely necessary. We need to go slower, one item at a time. And also, as you said, need to try with a supported Xcode/SDK combination.

[Rob Buis]

Right, do not take this patch too seriously, it is just something quick I did to build and do my normal development (I only put serious patches for review). At least it should show where the problems are.

As usual quicker is better, but I am not completely blocked. It is just an extra hassle when developing, and also it is not so nice to work on this stuff for me because touching some of these files (like PlatformHave.h) cause huge rebuilds on my poor 2015 mbp...

[Rob Buis]

(In reply to Saagar Jha from comment #4)
> I'm using the Xcode 12 SDK and seeing issues as well. The first one I ran
> into is a missing AXSIsolatedTreeMode, which I suspect is defined in the
> internal SDK and hiding the build failure:
> https://bugs.webkit.org/show_bug.cgi?id=208691#c19. I tried working around
> this by patching out ENABLE(ACCESSIBILITY_ISOLATED_TREE) code and ran into
> this, which I fixed: https://bugs.webkit.org/show_bug.cgi?id=213911. So now
> I'm up to WebCore, which fails because it's missing ANGLE headers. It seems
> to me like the "Copy Headers" phase from the ANGLE static library target
> should be doing this, but it doesn't show up in the build log at all so this
> might be some sort of dependency tracking failure in the build system.

We are in the same boat, although I run into AXSIsolatedTreeMode problem but not ANGLE problem it seems.

[Saagar Jha]

I hit that one only after patching out that feature. But the fact that you didn't see it in Xcode 11.5 lends credence to the fact that it's an Xcode-related change, and I wasn't just missing something stupid when I was looking at why that part was failing :P

I've also just found out that second beta has dropped, so I'll install that and see if that resolves any issues.

[Rob Buis]

(In reply to Alexey Proskuryakov from comment #3)
> I think that it would be easiest for someone at Apple to fix the build. Do
> you need it soon?

I agree that it would be easiest for someone at Apple to fix the build, as I said on my 2015 mbp working on this is no fun and I am not so familiar with the internal/public SDK. For me this is mid-priority as I have non WebKit projects I can work on.

[Rob Buis]

(In reply to Saagar Jha from comment #8)
> I've also just found out that second beta has dropped, so I'll install that
> and see if that resolves any issues.

Argh, 30 minutes after I grabbed 12A6159 :) Thanks for the heads up, I'll also try it.

[Alexey Proskuryakov]

I don't expect the second beta to change anything in this regard.

[Saagar Jha]

As you guessed, installing the new Xcode beta does not help :(

[Rob Buis]

Created attachment 404053 [details]
Patch

[Darin Adler]

Comment on attachment 404053 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=404053&action=review

> Source/WebKit/ChangeLog:9
> +        The preconnect ping feature does not build using the Xcode 11.5 public
> +        SDK, so add a check for APPLE_INTERNAL_SDK.

Could you tell me more about why it does not build using the public SDK? We should be very reluctant to put things behind USE(APPLE_INTERNAL_SDK).

[Alexey Proskuryakov]

Comment on attachment 404053 [details]
Patch

This CFNetwork functionality is not present in current macOS 11 seed versions, so it's more than an SDK issue. I don't think that we should wait for it to become available, so disabling it completely seems unavoidable. But we need to also compile out code in TestWebKitAPI. 

I can think of two approaches. 

1. Disable HAVE_PRECONNECT_PING in PlatformHave.h for public SDK for now. This would need to reference a bug to re-enable it.

2. Use runtime respondsToSelector checks (TestWebKitAPI part could be somewhat tricky).

[Alexey Proskuryakov]

> Actually it seems Xcode 11.5 is not supposed to work on Big Sur, at least the command line tools refuse to install.

Removed Xcode 11.5 from the title.

[Rob Buis]

(In reply to Darin Adler from comment #14)
> Comment on attachment 404053 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=404053&action=review
> 
> > Source/WebKit/ChangeLog:9
> > +        The preconnect ping feature does not build using the Xcode 11.5 public
> > +        SDK, so add a check for APPLE_INTERNAL_SDK.
> 
> Could you tell me more about why it does not build using the public SDK? We
> should be very reluctant to put things behind USE(APPLE_INTERNAL_SDK).

FWIW build error is:
In file included from WebKitBuild/Release/DerivedSources/WebCore/unified-sources/UnifiedSource198.cpp:5:
In file included from ./loader/CookieJar.cpp:35:
In file included from Source/WebCore/platform/network/NetworkStorageSession.h:42:
PAL/pal/spi/cf/CFNetworkSPI.h:71:1: error: expected unqualified-id
@interface _NSHTTPConnectionInfo : NSObject
^
PAL/pal/spi/cf/CFNetworkSPI.h:73:1: error: expected unqualified-id
@property (readonly) BOOL isValid;
^
PAL/pal/spi/cf/CFNetworkSPI.h:74:1: error: expected unqualified-id
@end
^
PAL/pal/spi/cf/CFNetworkSPI.h:78:1: error: expected unqualified-id
@end
^

[Darin Adler]

Comment on attachment 404053 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=404053&action=review

>>> Source/WebKit/ChangeLog:9
>>> +        SDK, so add a check for APPLE_INTERNAL_SDK.
>> 
>> Could you tell me more about why it does not build using the public SDK? We should be very reluctant to put things behind USE(APPLE_INTERNAL_SDK).
> 
> FWIW build error is:
> In file included from WebKitBuild/Release/DerivedSources/WebCore/unified-sources/UnifiedSource198.cpp:5:
> In file included from ./loader/CookieJar.cpp:35:
> In file included from Source/WebCore/platform/network/NetworkStorageSession.h:42:
> PAL/pal/spi/cf/CFNetworkSPI.h:71:1: error: expected unqualified-id
> @interface _NSHTTPConnectionInfo : NSObject
> ^
> PAL/pal/spi/cf/CFNetworkSPI.h:73:1: error: expected unqualified-id
> @property (readonly) BOOL isValid;
> ^
> PAL/pal/spi/cf/CFNetworkSPI.h:74:1: error: expected unqualified-id
> @end
> ^
> PAL/pal/spi/cf/CFNetworkSPI.h:78:1: error: expected unqualified-id
> @end
> ^

That error above is about Objective-C declarations in a header that’s included by a non-Objective-C source file. The fix for that one error is to put the declarations inside an #ifdef __OBJC__ or defined(__OBJC__) block. Note, for example, NSURLSessionTask below.

[Rob Buis]

(In reply to Darin Adler from comment #18)

> 
> That error above is about Objective-C declarations in a header that’s
> included by a non-Objective-C source file. The fix for that one error is to
> put the declarations inside an #ifdef __OBJC__ or defined(__OBJC__) block.
> Note, for example, NSURLSessionTask below.

Thanks, I moved that fix to https://bugs.webkit.org/show_bug.cgi?id=214238.

[Saagar Jha]

Couple new issues just cropped up on 11.0 having to do with HAVE(LARGE_CONTROL_SIZE) (corresponding availability macro defined anywhere) and HAVE(ALTERNATE_ICONS) (defined correctly and enabled on my system, but causes the use of what looks like undeclared SPI +[NSImage _imageWithSystemSymbolName:]). I'll try declaring the appropriate things, seeing if that helps, and sending in a patch if so.

[Darin Adler]

When new versions come out there are always a ton of things like this, and we know they exist for Big Sur, and the new versions of iOS, watchOS and tvOS.

People at Apple typically fix these during the summer while the operating systems make their way from beta to GM.

In this case, Jonathan Bedard said (in another bug), "Aakash started work on [macOS Big Sur 2 weeks ago, but I think tvOS and watchOS EWS were a higher priority. Someone from [the Apple WebKit team] will get [the macOS Big Sur build] working in the next 2 weeks."

[Saagar Jha]

(I appreciate the update, but am a bit unsure if that was meant to be purely informational or a polite way of saying "we know about the issue, we're working on fixing it, please stop bothering us with updates and don't send in patches"…)

[Darin Adler]

I don’t mind if some non-Apple contributors send patches, but there’s a good chance they won’t be correct, and I hope the process is not too frustrating.

I also don’t think this can all be completed by someone outside Apple.

And I wanted to point out that we plan to handle it this year the way we have every other year.

[Saagar Jha]

Fair enough :) I've created https://bugs.webkit.org/show_bug.cgi?id=214338 with a patch that has a slight chance of having useful things in it. That, coupled with disabling ENABLE_ACCESSIBILITY_ISOLATED_TREE and HAVE_INCREMENTAL_PDF_APIS (which still use SPI that I didn't want to bother guessing the values of) got me a successful build for Intel, though I haven't tried running it yet. On Apple silicon I get through most of the build but the linker fails, so I still need to figure out what is going on there.

[Rob Buis]

(In reply to Saagar Jha from comment #24)
> Fair enough :) I've created https://bugs.webkit.org/show_bug.cgi?id=214338
> with a patch that has a slight chance of having useful things in it. That,
> coupled with disabling ENABLE_ACCESSIBILITY_ISOLATED_TREE and
> HAVE_INCREMENTAL_PDF_APIS (which still use SPI that I didn't want to bother
> guessing the values of) got me a successful build for Intel, though I
> haven't tried running it yet. On Apple silicon I get through most of the
> build but the linker fails, so I still need to figure out what is going on
> there.

It seems we are doing similar work, see the bugs linked to this one :)
I think it is not needed to disable ENABLE_ACCESSIBILITY_ISOLATED_TREE, see https://bugs.webkit.org/show_bug.cgi?id=214237.
Quite possibly the HAVE_INCREMENTAL_PDF_APIS issue is one that only Apple employees can fix.

[Jonathan Bedard]

Linked this up to https://bugs.webkit.org/show_bug.cgi?id=214396, which should be enough to get WebKit to build for Big Sur with the Seed 1 SDK.

[Rob Buis]

(In reply to Jonathan Bedard from comment #26)
> Linked this up to https://bugs.webkit.org/show_bug.cgi?id=214396, which
> should be enough to get WebKit to build for Big Sur with the Seed 1 SDK.

That fixes my build, thanks! I will close this bug therefore.