212436 2020-05-27 15:00:26 -0700 REGRESSION (r262203): Build errors during JavaScriptCore "Generate Derived Sources" about missing Availability.h header and invalid SDKROOT [-Wmissing-sysroot] 2020-05-28 10:50:59 -0700 1 1 1 Unclassified WebKit JavaScriptCore Other Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=212451 https://bugs.webkit.org/show_bug.cgi?id=109324 InRadar P2 Normal --- 210014 1 ddkilzer ddkilzer bburg darin ews-watchlist hi jlewis3 keith_miller krollin mark.lam msaboff ryanhaddad saam tzagallo webkit-bug-importer oldest_to_newest 1656604 0 ddkilzer 2020-05-27 15:00:26 -0700 Build errors during JavaScriptCore "Generate Derived Sources" about missing Availability.h header and invalid SDKROOT [-Wmissing-sysroot]. [...] === BUILD AGGREGATE TARGET Derived Sources OF PROJECT JavaScriptCore WITH CONFIGURATION Debug === Check dependencies PhaseScriptExecution Generate\ Derived\ Sources /var/build/JavaScriptCore.build/Debug/Derived\ Sources.build/Script-65FB3F6509D11E9100F49DEB.sh cd /Users/ddkilzer/Data/WebKit.git/Source/JavaScriptCore /bin/sh -c \"/var/build/JavaScriptCore.build/Debug/Derived\ Sources.build/Script-65FB3F6509D11E9100F49DEB.sh\" clang: warning: no such sysroot directory: 'macosx10.14internal' [-Wmissing-sysroot] In file included from <built-in>:1: In file included from /var/build/Debug/usr/local/include/wtf/Platform.h:44: /var/build/Debug/usr/local/include/wtf/PlatformOS.h:36:10: fatal error: 'Availability.h' file not found #include <Availability.h> ^~~~~~~~~~~~~~~~ 1 error generated. clang: warning: no such sysroot directory: 'macosx10.14internal' [-Wmissing-sysroot] In file included from <built-in>:1: In file included from /var/build/Debug/usr/local/include/wtf/Platform.h:44: /var/build/Debug/usr/local/include/wtf/PlatformOS.h:36:10: fatal error: 'Availability.h' file not found #include <Availability.h> ^~~~~~~~~~~~~~~~ 1 error generated. clang: warning: no such sysroot directory: 'macosx10.14internal' [-Wmissing-sysroot] In file included from <built-in>:1: In file included from /var/build/Debug/usr/local/include/wtf/Platform.h:44: /var/build/Debug/usr/local/include/wtf/PlatformOS.h:36:10: fatal error: 'Availability.h' file not found #include <Availability.h> ^~~~~~~~~~~~~~~~ 1 error generated. clang: warning: no such sysroot directory: 'macosx10.14internal' [-Wmissing-sysroot] In file included from <built-in>:1: In file included from /var/build/Debug/usr/local/include/wtf/Platform.h:44: /var/build/Debug/usr/local/include/wtf/PlatformOS.h:36:10: fatal error: 'Availability.h' file not found #include <Availability.h> ^~~~~~~~~~~~~~~~ 1 error generated. python JavaScriptCore/Scripts/UpdateContents.py 'JavaScriptCore/builtins/AsyncFromSyncIteratorPrototype.js JavaScriptCore/builtins/ArrayConstructor.js JavaScriptCore/builtins/ArrayIteratorPrototype.js JavaScriptCore/builtins/ArrayPrototype.js JavaScriptCore/builtins/AsyncIteratorPrototype.js JavaScriptCore/builtins/AsyncFunctionPrototype.js JavaScriptCore/builtins/AsyncGeneratorPrototype.js JavaScriptCore/builtins/DatePrototype.js JavaScriptCore/builtins/FunctionPrototype.js JavaScriptCore/builtins/GeneratorPrototype.js JavaScriptCore/builtins/GlobalObject.js JavaScriptCore/builtins/GlobalOperations.js JavaScriptCore/builtins/InspectorInstrumentationObject.js JavaScriptCore/builtins/InternalPromiseConstructor.js JavaScriptCore/builtins/IteratorHelpers.js JavaScriptCore/builtins/IteratorPrototype.js JavaScriptCore/builtins/MapIteratorPrototype.js JavaScriptCore/builtins/MapPrototype.js JavaScriptCore/builtins/ModuleLoader.js JavaScriptCore/builtins/NumberConstructor.js JavaScriptCore/builtins/ObjectConstructor.js JavaScriptCore/builtins/PromiseConstructor.js JavaScriptCore/builtins/PromiseOperations.js JavaScriptCore/builtins/PromisePrototype.js JavaScriptCore/builtins/ReflectObject.js JavaScriptCore/builtins/RegExpPrototype.js JavaScriptCore/builtins/RegExpStringIteratorPrototype.js JavaScriptCore/builtins/SetIteratorPrototype.js JavaScriptCore/builtins/SetPrototype.js JavaScriptCore/builtins/StringConstructor.js JavaScriptCore/builtins/StringIteratorPrototype.js JavaScriptCore/builtins/StringPrototype.js JavaScriptCore/builtins/TypedArrayConstructor.js JavaScriptCore/builtins/TypedArrayPrototype.js JavaScriptCore/builtins/WebAssembly.js JavaScriptCore/Scripts/wkbuiltins/__init__.py JavaScriptCore/Scripts/wkbuiltins/wkbuiltins.py JavaScriptCore/Scripts/wkbuiltins/builtins_generator.py JavaScriptCore/Scripts/wkbuiltins/builtins_model.py JavaScriptCore/Scripts/wkbuiltins/builtins_templates.py JavaScriptCore/Scripts/wkbuiltins/builtins_generate_combined_header.py JavaScriptCore/Scripts/wkbuiltins/builtins_generate_combined_implementation.py JavaScriptCore/Scripts/wkbuiltins/builtins_generate_separate_header.py JavaScriptCore/Scripts/wkbuiltins/builtins_generate_separate_implementation.py JavaScriptCore/Scripts/wkbuiltins/builtins_generate_wrapper_header.py JavaScriptCore/Scripts/wkbuiltins/builtins_generate_wrapper_implementation.py JavaScriptCore/Scripts/wkbuiltins/builtins_generate_internals_wrapper_header.py JavaScriptCore/Scripts/wkbuiltins/builtins_generate_internals_wrapper_implementation.py JavaScriptCore/Scripts/generate-js-builtins.py JavaScriptCore/Scripts/lazywriter.py ' JavaScriptCore_BUILTINS_DEPENDENCIES_LIST python JavaScriptCore/Scripts/UpdateContents.py 'JavaScriptCore/inspector/protocol/Animation.json JavaScriptCore/inspector/protocol/ApplicationCache.json JavaScriptCore/inspector/protocol/Audit.json JavaScriptCore/inspector/protocol/Browser.json JavaScriptCore/inspector/protocol/CPUProfiler.json JavaScriptCore/inspector/protocol/CSS.json JavaScriptCore/inspector/protocol/Canvas.json JavaScriptCore/inspector/protocol/Console.json JavaScriptCore/inspector/protocol/DOM.json JavaScriptCore/inspector/protocol/DOMDebugger.json JavaScriptCore/inspector/protocol/DOMStorage.json JavaScriptCore/inspector/protocol/Database.json JavaScriptCore/inspector/protocol/Debugger.json JavaScriptCore/inspector/protocol/GenericTypes.json JavaScriptCore/inspector/protocol/Heap.json JavaScriptCore/inspector/protocol/IndexedDB.json JavaScriptCore/inspector/protocol/Inspector.json JavaScriptCore/inspector/protocol/LayerTree.json JavaScriptCore/inspector/protocol/Memory.json JavaScriptCore/inspector/protocol/Network.json JavaScriptCore/inspector/protocol/Page.json JavaScriptCore/inspector/protocol/Recording.json JavaScriptCore/inspector/protocol/Runtime.json JavaScriptCore/inspector/protocol/ScriptProfiler.json JavaScriptCore/inspector/protocol/Security.json JavaScriptCore/inspector/protocol/ServiceWorker.json JavaScriptCore/inspector/protocol/Target.json JavaScriptCore/inspector/protocol/Timeline.json JavaScriptCore/inspector/protocol/Worker.json ' EnabledInspectorDomains Command /bin/sh emitted errors but did not return a nonzero exit code to indicate failure [...] 1656606 1 ddkilzer 2020-05-27 15:02:09 -0700 This fixes it: $ git diff --cached diff --git a/Source/JavaScriptCore/Scripts/generate-derived-sources.sh b/Source/JavaScriptCore/Scripts/generate-derived-sources.sh index 2414765431d..17939f2c920 100755 --- a/Source/JavaScriptCore/Scripts/generate-derived-sources.sh +++ b/Source/JavaScriptCore/Scripts/generate-derived-sources.sh @@ -15,4 +15,4 @@ if [ ! $CC ]; then export CC="`xcrun -find clang`" fi -make --no-builtin-rules -f "JavaScriptCore/DerivedSources.make" -j `/usr/sbin/sysctl -n hw.ncpu` "${ARGS[@]}" +make --no-builtin-rules -f "JavaScriptCore/DerivedSources.make" -j `/usr/sbin/sysctl -n hw.ncpu` SDKROOT="${SDKROOT}" "${ARGS[@]}" Will check in momentarily. Building with Apple internal SDK on 10.14.6. 1656608 2 ddkilzer 2020-05-27 15:03:15 -0700 I suspect this issue may have always been present, but it didn't matter previously because the <Availability.h> header wasn't being included during this step before. 1656610 3 ddkilzer 2020-05-27 15:07:50 -0700 I was building using these commands (both of which worked until recently): make debug SDKROOT=macosx.internal make debug SDKROOT=macosx10.14internal Maybe a regression from Bug 209890 (untested): Enable the use of XCBuild by default in Apple builds <https://bugs.webkit.org/show_bug.cgi?id=209890> 1656611 4 400386 ddkilzer 2020-05-27 15:09:13 -0700 Created attachment 400386 Patch v1 1656612 5 ddkilzer 2020-05-27 15:09:54 -0700 Letting EWS chew on this before landing an unreviewed build fix. 1656613 6 webkit-bug-importer 2020-05-27 15:13:03 -0700 <rdar://problem/63692399> 1656641 7 krollin 2020-05-27 15:42:49 -0700 Just curious, what changed such that <Availability.h> is now being pulled in? Also, I would think that SDKROOT is already exported. Why does defining it on the command line work? 1656645 8 400386 ddkilzer 2020-05-27 15:48:34 -0700 Comment on attachment 400386 Patch v1 All Apple OS targets built, and I reviewed build logs and didn't see any errors on the build phase script that changed. Marking cq+. 1656651 9 ddkilzer 2020-05-27 15:54:28 -0700 (In reply to Keith Rollin from comment #7) > Just curious, what changed such that <Availability.h> is now being pulled in? I don't know the answer to this because I've been prioritizing a fix to be committed first. Will try to answer if I have time today. > Also, I would think that SDKROOT is already exported. Why does defining it > on the command line work? During debugging I printed out all of `env` in the script (and changed the script to add -x to the #! line to see all the environment variables). In that case, the SDKROOT environment variable was a fully qualified path to the SDK in /Applications/Xcode.app/Contents/.../. I honestly don't know where JavaScriptCore/DerivedSources.make was getting the original Makefile argument from as it didn't show up as part of "${ARGS[@]}". I assumed it was some issue with Xcode that I couldn't reason about, and passing the argument explicitly fixed it, so that's the change I made. 1656652 10 ddkilzer 2020-05-27 15:55:43 -0700 (In reply to David Kilzer (:ddkilzer) from comment #9) > (In reply to Keith Rollin from comment #7) > > Just curious, what changed such that <Availability.h> is now being pulled in? > > I don't know the answer to this because I've been prioritizing a fix to be > committed first. Will try to answer if I have time today. > > > Also, I would think that SDKROOT is already exported. Why does defining it > > on the command line work? > > During debugging I printed out all of `env` in the script (and changed the > script to add -x to the #! line to see all the environment variables). In > that case, the SDKROOT environment variable was a fully qualified path to > the SDK in /Applications/Xcode.app/Contents/.../. > > I honestly don't know where JavaScriptCore/DerivedSources.make was getting > the original Makefile argument from as it didn't show up as part of > "${ARGS[@]}". I assumed it was some issue with Xcode that I couldn't reason > about, and passing the argument explicitly fixed it, so that's the change I > made. Defining it on the command-line works because Source/JavaScriptCore/DerivedSources.make does this: ifneq ($(SDKROOT),) SDK_FLAGS=-isysroot $(SDKROOT) endif So the SDKROOT I pass in on the command-line overrides whatever value it was picking up before. 1656656 11 ews-feeder 2020-05-27 16:02:35 -0700 Committed r262222: <https://trac.webkit.org/changeset/262222> All reviewed patches have been landed. Closing bug and clearing flags on attachment 400386. 1656669 12 ddkilzer 2020-05-27 16:25:15 -0700 Using `git bisect`, I found the regression was the fix for Bug 210014 in r262203: Web Inspector: replace `featureGuard` and `availability` with a combined `condition` that accepts any macro https://bugs.webkit.org/show_bug.cgi?id=210014 https://trac.webkit.org/r262203 That added all this code, which wasn't there before: +ifneq ($(SDKROOT),) + SDK_FLAGS=-isysroot $(SDKROOT) +endif Which were then used in these new commands (which didn't exist before): +ifeq ($(shell $(CC) -std=gnu++1z -x c++ -E -P -dM $(SDK_FLAGS) $(TARGET_TRIPLE_FLAGS) $(FRAMEWORK_FLAGS) $(HEADER_FLAGS) -include "wtf/Platform.h" /d ev/null | grep ' WTF_PLATFORM_MAC ' | cut -d' ' -f3), 1) + PREPROCESSOR_DEFINES += WTF_PLATFORM_MAC +endif + +ifeq ($(shell $(CC) -std=gnu++1z -x c++ -E -P -dM $(SDK_FLAGS) $(TARGET_TRIPLE_FLAGS) $(FRAMEWORK_FLAGS) $(HEADER_FLAGS) -include "wtf/Platform.h" /d ev/null | grep ' WTF_PLATFORM_IOS_FAMILY ' | cut -d' ' -f3), 1) + PREPROCESSOR_DEFINES += WTF_PLATFORM_IOS_FAMILY +endif + +ifeq ($(shell $(CC) -std=gnu++1z -x c++ -E -P -dM $(SDK_FLAGS) $(TARGET_TRIPLE_FLAGS) $(FRAMEWORK_FLAGS) $(HEADER_FLAGS) -include "wtf/Platform.h" /d ev/null | grep ' USE_CF ' | cut -d' ' -f3), 1) + PREPROCESSOR_DEFINES += USE_CF +endif + +ifeq ($(shell $(CC) -std=gnu++1z -x c++ -E -P -dM $(SDK_FLAGS) $(TARGET_TRIPLE_FLAGS) $(FRAMEWORK_FLAGS) $(HEADER_FLAGS) -include "wtf/Platform.h" /d ev/null | grep HAVE_OS_DARK_MODE_SUPPORT | cut -d' ' -f3), 1) + PREPROCESSOR_DEFINES += HAVE_OS_DARK_MODE_SUPPORT +else I'm still not sure why DerivedSources.make was using the SDKROOT that I passed in on the `make debug SDKROOT=...` command-line, but I think the fix was correct. 1656916 13 ddkilzer 2020-05-28 10:50:45 -0700 Another point of interest: Source/WebCore/Scripts/generate-derived-sources.sh also includes the SDKROOT="${SDKROOT}" argument when running a command on Source/WebCore/DerivedSources.make. It has been present since Source/WebCore/Scripts/generate-derived-sources.sh was a build phase script contained within WebCore.xcodeproj/project.pbxproj, and it was added in r142546: Bug 109324: Make WebCore Derived Sources work with SDK identifiers too <https://bugs.webkit.org/show_bug.cgi?id=109324> <https://trac.webkit.org/r142546> 400386 2020-05-27 15:09:13 -0700 2020-05-27 16:02:36 -0700 Patch v1 bug-212436-20200527151053.patch text/plain 1543 ddkilzer U3VidmVyc2lvbiBSZXZpc2lvbjogMjYyMjE3CmRpZmYgLS1naXQgYS9Tb3VyY2UvSmF2YVNjcmlw dENvcmUvQ2hhbmdlTG9nIGIvU291cmNlL0phdmFTY3JpcHRDb3JlL0NoYW5nZUxvZwppbmRleCA4 NDY1YmU1YmE4YTJkYWE3ZTg3ZTE0YzJjMmNkZmQ3ZDdjMzY4YzZjLi4xNTQ4OTU1ZjhlOWQwOWFm ODNkOTU4NDIxMGFmOWUxNDQyYWI5ZmQyIDEwMDY0NAotLS0gYS9Tb3VyY2UvSmF2YVNjcmlwdENv cmUvQ2hhbmdlTG9nCisrKyBiL1NvdXJjZS9KYXZhU2NyaXB0Q29yZS9DaGFuZ2VMb2cKQEAgLTEs MyArMSwxMyBAQAorMjAyMC0wNS0yNyAgRGF2aWQgS2lsemVyICA8ZGRraWx6ZXJAYXBwbGUuY29t PgorCisgICAgICAgIFJFR1JFU1NJT046IEJ1aWxkIGVycm9ycyBkdXJpbmcgSmF2YVNjcmlwdENv cmUgIkdlbmVyYXRlIERlcml2ZWQgU291cmNlcyIgYWJvdXQgbWlzc2luZyBBdmFpbGFiaWxpdHku aCBoZWFkZXIgYW5kIGludmFsaWQgU0RLUk9PVCBbLVdtaXNzaW5nLXN5c3Jvb3RdCisgICAgICAg IDxodHRwczovL3dlYmtpdC5vcmcvYi8yMTI0MzY+CisKKyAgICAgICAgVW5yZXZpZXdlZCBidWls ZCBmaXguCisKKyAgICAgICAgKiBTY3JpcHRzL2dlbmVyYXRlLWRlcml2ZWQtc291cmNlcy5zaDog UGFzcyBTREtST09UIHRvIGBtYWtlYAorICAgICAgICBjb21tYW5kIGZyb20gdGhlIGN1cnJlbnQg ZW52aXJvbm1lbnQuCisKIDIwMjAtMDUtMjcgIERldmluIFJvdXNzbyAgPGRyb3Vzc29AYXBwbGUu Y29tPgogCiAgICAgICAgIFdlYiBJbnNwZWN0b3I6IHJlcGxhY2UgYGZlYXR1cmVHdWFyZGAgYW5k IGBhdmFpbGFiaWxpdHlgIHdpdGggYSBjb21iaW5lZCBgY29uZGl0aW9uYCB0aGF0IGFjY2VwdHMg YW55IG1hY3JvCmRpZmYgLS1naXQgYS9Tb3VyY2UvSmF2YVNjcmlwdENvcmUvU2NyaXB0cy9nZW5l cmF0ZS1kZXJpdmVkLXNvdXJjZXMuc2ggYi9Tb3VyY2UvSmF2YVNjcmlwdENvcmUvU2NyaXB0cy9n ZW5lcmF0ZS1kZXJpdmVkLXNvdXJjZXMuc2gKaW5kZXggMjQxNDc2NTQzMWQ5YmI1ZDQxY2U0ODQw NTliOWY5NTU5NGRlZWE3MC4uMTc5MzlmMmM5MjAxY2NiMmQ2ZWVmYjdlOTI3NjJhOGYxZDc4YzA3 YyAxMDA3NTUKLS0tIGEvU291cmNlL0phdmFTY3JpcHRDb3JlL1NjcmlwdHMvZ2VuZXJhdGUtZGVy aXZlZC1zb3VyY2VzLnNoCisrKyBiL1NvdXJjZS9KYXZhU2NyaXB0Q29yZS9TY3JpcHRzL2dlbmVy YXRlLWRlcml2ZWQtc291cmNlcy5zaApAQCAtMTUsNCArMTUsNCBAQCBpZiBbICEgJENDIF07IHRo ZW4KICAgICBleHBvcnQgQ0M9ImB4Y3J1biAtZmluZCBjbGFuZ2AiCiBmaQogCi1tYWtlIC0tbm8t YnVpbHRpbi1ydWxlcyAtZiAiSmF2YVNjcmlwdENvcmUvRGVyaXZlZFNvdXJjZXMubWFrZSIgLWog YC91c3Ivc2Jpbi9zeXNjdGwgLW4gaHcubmNwdWAgIiR7QVJHU1tAXX0iCittYWtlIC0tbm8tYnVp bHRpbi1ydWxlcyAtZiAiSmF2YVNjcmlwdENvcmUvRGVyaXZlZFNvdXJjZXMubWFrZSIgLWogYC91 c3Ivc2Jpbi9zeXNjdGwgLW4gaHcubmNwdWAgU0RLUk9PVD0iJHtTREtST09UfSIgIiR7QVJHU1tA XX0iCg==