Bug 229122 – Fix ASan+UBSan builds with Xcode 13

• Component: Tools / Tests
• Status: RESOLVED FIXED

[David Kilzer (:ddkilzer)]

Fix ASan+UBSan builds with Xcode 13.

The xcodebuild command from Xcode 13 does not support multiple -xcconfig switches on the command-line, so create Tools/sanitizer/asan+ubsan.xcconfig and use that when both ASan and UBSan are enabled.

[David Kilzer (:ddkilzer)]

Created attachment 435566 [details]
Patch v1

[David Kilzer (:ddkilzer)]

Note that I didn't try to come up with a general solution for including multiple *.xcconfig files because I'm not sure if there are any other combinations that might be useful (or even used).

[Alexey Proskuryakov]

Comment on attachment 435566 [details]
Patch v1

Is this an intentional change in Xcode, or a bug?

[Alexey Proskuryakov]

Comment on attachment 435566 [details]
Patch v1

View in context: https://bugs.webkit.org/attachment.cgi?id=435566&action=review

> Tools/sanitizer/asan+ubsan.xcconfig:2
> +#include "asan.xcconfig"
> +#include "ubsan.xcconfig"

I'm a nit uneasy about including sanitizer.xcconfig twice though. Did you confirm that nothing weird happens?

[David Kilzer (:ddkilzer)]

(In reply to Alexey Proskuryakov from comment #4)
> Comment on attachment 435566 [details]
> Patch v1
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=435566&action=review
> 
> > Tools/sanitizer/asan+ubsan.xcconfig:2
> > +#include "asan.xcconfig"
> > +#include "ubsan.xcconfig"
> 
> I'm a nit uneasy about including sanitizer.xcconfig twice though. Did you
> confirm that nothing weird happens?

The error message seems pretty explicit about it:

$ ./Tools/Scripts/set-webkit-configuration --reset
$ ./Tools/Scripts/set-webkit-configuration --asan --ubsan
$ make release
Current settings: Configuration:Release Arch:x86_64 ASan UBSan
( if [[ -t 1 ]]; then COLOR_DIAGNOSTICS_ARG="COLOR_DIAGNOSTICS=YES"; fi; xcodebuild  `perl -I../../../OpenSource/Tools/Scripts -Mwebkitdirs -e 'print XcodeOptionString()'` ${COLOR_DIAGNOSTICS_ARG}   -UseNewBuildSystem=NO -hideShellScriptEnvironment  | cat && exit ${PIPESTATUS[0]} )
xcodebuild: error: option '-xcconfig' may only be provided once

Usage: xcodebuild [-project <projectname>] [[-target <targetname>]...|-alltargets] [-configuration <configurationname>] [-arch <architecture>]... [-sdk [<sdkname>|<sdkpath>]] [-showBuildSettings [-json]] [<buildsetting>=<value>]... [<buildaction>]...
       xcodebuild [-project <projectname>] -scheme <schemeName> [-destination <destinationspecifier>]... [-configuration <configurationname>] [-arch <architecture>]... [-sdk [<sdkname>|<sdkpath>]] [-showBuildSettings [-json]] [-showdestinations] [<buildsetting>=<value>]... [<buildaction>]...
       xcodebuild -workspace <workspacename> -scheme <schemeName> [-destination <destinationspecifier>]... [-configuration <configurationname>] [-arch <architecture>]... [-sdk [<sdkname>|<sdkpath>]] [-showBuildSettings] [-showdestinations] [<buildsetting>=<value>]... [<buildaction>]...
       xcodebuild -version [-sdk [<sdkfullpath>|<sdkname>] [-json] [<infoitem>] ]
       xcodebuild -list [[-project <projectname>]|[-workspace <workspacename>]] [-json]
       xcodebuild -showsdks [-json]
       xcodebuild -exportArchive -archivePath <xcarchivepath> [-exportPath <destinationpath>] -exportOptionsPlist <plistpath>
       xcodebuild -exportNotarizedApp -archivePath <xcarchivepath> -exportPath <destinationpath>
       xcodebuild -exportLocalizations -localizationPath <path> -project <projectname> [-exportLanguage <targetlanguage>...[-includeScreenshots]]
       xcodebuild -importLocalizations -localizationPath <path> -project <projectname> [-mergeImport]
       xcodebuild -resolvePackageDependencies [-project <projectname>|-workspace <workspacename>] -clonedSourcePackagesDirPath <path>
       xcodebuild -create-xcframework [-help] [-framework <path>] [-library <path> [-headers <path>]] -output <path>

Options:
    -usage                                                   print brief usage
    -help                                                    print complete usage
    -verbose                                                 provide additional status output
    -license                                                 show the Xcode and SDK license agreements
    -checkFirstLaunchStatus                                  Check if any First Launch tasks need to be performed
    -runFirstLaunch                                          install packages and agree to the license
    -project NAME                                            build the project NAME
    -target NAME                                             build the target NAME
    -alltargets                                              build all targets
    -workspace NAME                                          build the workspace NAME
    -scheme NAME                                             build the scheme NAME
    -configuration NAME                                      use the build configuration NAME for building each target
    -xcconfig PATH                                           apply the build settings defined in the file at PATH as overrides
    -arch ARCH                                               build each target for the architecture ARCH; this will override architectures defined in the project
    -sdk SDK                                                 use SDK as the name or path of the base SDK when building the project
    -toolchain NAME                                          use the toolchain with identifier or name NAME
    -destination DESTINATIONSPECIFIER                        use the destination described by DESTINATIONSPECIFIER (a comma-separated set of key=value pairs describing the destination to use)
    -destination-timeout TIMEOUT                             wait for TIMEOUT seconds while searching for the destination device
    -parallelizeTargets                                      build independent targets in parallel
    -jobs NUMBER                                             specify the maximum number of concurrent build operations
    -maximum-concurrent-test-device-destinations NUMBER      the maximum number of device destinations to test on concurrently
    -maximum-concurrent-test-simulator-destinations NUMBER   the maximum number of simulator destinations to test on concurrently
    -parallel-testing-enabled YES|NO                         overrides the per-target setting in the scheme
    -parallel-testing-worker-count NUMBER                    the exact number of test runners that will be spawned during parallel testing
    -maximum-parallel-testing-workers NUMBER                 the maximum number of test runners that will be spawned during parallel testing
    -dry-run                                                 do everything except actually running the commands
    -quiet                                                   do not print any output except for warnings and errors
    -hideShellScriptEnvironment                              don't show shell script environment variables in build log
    -showsdks                                                display a compact list of the installed SDKs
    -showdestinations                                        display a list of destinations
    -showTestPlans                                           display a list of test plans
    -showBuildSettings                                       display a list of build settings and values
    -showBuildSettingsForIndex                               display build settings for the index service
    -list                                                    lists the targets and configurations in a project, or the schemes in a workspace
    -find-executable NAME                                    display the full path to executable NAME in the provided SDK and toolchain
    -find-library NAME                                       display the full path to library NAME in the provided SDK and toolchain
    -version                                                 display the version of Xcode; with -sdk will display info about one or all installed SDKs
    -enableAddressSanitizer YES|NO                           turn the address sanitizer on or off
    -enableThreadSanitizer YES|NO                            turn the thread sanitizer on or off
    -enableUndefinedBehaviorSanitizer YES|NO                 turn the undefined behavior sanitizer on or off
    -resultBundlePath PATH                                   specifies the directory where a result bundle describing what occurred will be placed
    -resultStreamPath PATH                                   specifies the file where a result stream will be written to (the file must already exist)
    -resultBundleVersion 3 [default]                         specifies which result bundle version should be used
    -clonedSourcePackagesDirPath PATH                        specifies the directory to which remote source packages are fetch or expected to be found
    -derivedDataPath PATH                                    specifies the directory where build products and other derived data will go
    -archivePath PATH                                        specifies the directory where any created archives will be placed, or the archive that should be exported
    -exportArchive                                           specifies that an archive should be exported
    -exportNotarizedApp                                      export an archive that has been notarized by Apple
    -exportOptionsPlist PATH                                 specifies a path to a plist file that configures archive exporting
    -enableCodeCoverage YES|NO                               turn code coverage on or off when testing
    -exportPath PATH                                         specifies the destination for the product exported from an archive
    -skipUnavailableActions                                  specifies that scheme actions that cannot be performed should be skipped instead of causing a failure
    -exportLocalizations                                     exports completed and outstanding project localizations
    -importLocalizations                                     imports localizations for a project, assuming any necessary localized resources have been created in Xcode
    -localizationPath                                        specifies a path to XLIFF localization files
    -exportLanguage                                          specifies multiple optional ISO 639-1 languages included in a localization export
    -xctestrun                                               specifies a path to a test run specification
    -enablePerformanceTestsDiagnostics YES|NO                enables performance trace and memgraph collection for performance XCTests
    -testPlan                                                specifies the name of the test plan associated with the scheme to use for testing
    -only-testing                                            constrains testing by specifying tests to include, and excluding other tests
    -only-testing:TEST-IDENTIFIER                            constrains testing by specifying tests to include, and excluding other tests
    -skip-testing                                            constrains testing by specifying tests to exclude, but including other tests
    -skip-testing:TEST-IDENTIFIER                            constrains testing by specifying tests to exclude, but including other tests
    -test-timeouts-enabled YES|NO                            enable or disable test timeout behavior
    -default-test-execution-time-allowance SECONDS           the default execution time an individual test is given to execute, if test timeouts are enabled
    -maximum-test-execution-time-allowance SECONDS           the maximum execution time an individual test is given to execute, regardless of the test's preferred allowance
    -test-iterations <number>                                If specified, tests will run <number> times. May be used in conjunction with either -retry-tests-on-failure or -run-tests-until-failure, in which case this will become the maximum number of iterations.
    -retry-tests-on-failure                                  If specified, tests will retry on failure. May be used in conjunction with -test-iterations <number>, in which case <number> will be the maximum number of iterations. Otherwise, a maximum of 3 is assumed. May not be used with -run-tests-until-failure.
    -run-tests-until-failure                                 If specified, tests will run until they fail. May be used in conjunction with -test-iterations <number>, in which case <number> will be the maximum number of iterations. Otherwise, a maximum of 100 is assumed. May not be used with -retry-tests-on-failure.
    -test-repetition-relaunch-enabled YES|NO                 Enable or disable, tests repeating in a new process for each repetition.  Must be used in conjunction with -test-iterations, -retry-tests-on-failure, or -run-tests-until-failure. If not specified, tests will repeat in the same process.
    -only-test-configuration                                 constrains testing by specifying test configurations to include, and excluding other test configurations
    -skip-test-configuration                                 constrains testing by specifying test configurations to exclude, but including other test configurations
    -testLanguage                                            constrains testing by specifying ISO 639-1 language in which to run the tests
    -testRegion                                              constrains testing by specifying ISO 3166-1 region in which to run the tests
    -resolvePackageDependencies                              resolves any Swift package dependencies referenced by the project or workspace
    -disableAutomaticPackageResolution                       prevents packages from automatically being resolved to versions other than those recorded in the `Package.resolved` file
    -disablePackageRepositoryCache                           disable use of a local cache of remote package repositories
    -packageCachePath                                        path of caches used for package support
    -json                                                    output as JSON (note: -json implies -quiet)
    -allowProvisioningUpdates                                Allow xcodebuild to communicate with the Apple Developer website. For automatically signed targets, xcodebuild will create and update profiles, app IDs, and certificates. For manually signed targets, xcodebuild will download missing or updated provisioning profiles. Requires a developer account to have been added in Xcode's Accounts preference pane or an App Store Connect authentication key to be specified via the -authenticationKeyPath, -authenticationKeyID, and -authenticationKeyIssuerID parameters.
    -allowProvisioningDeviceRegistration                     Allow xcodebuild to register your destination device on the developer portal if necessary. This flag only takes effect if -allowProvisioningUpdates is also passed.
    -authenticationKeyPath                                   specifies the path to an authentication key issued by App Store Connect. If specified, xcodebuild will authenticate with the Apple Developer website using this credential. The -authenticationKeyID and -authenticationKeyIssuerID parameters are required.
    -authenticationKeyID                                     specifies the key identifier associated with the App Store Conect authentication key at -authenticationKeyPath. This string can be located in the users and access details for your provider at "https://appstoreconnect.apple.com".
    -authenticationKeyIssuerID                               specifies the App Store Connect issuer identifier associated with the authentication key at -authenticationKeyPath. This string can be located in the users and access details for your provider at "https://appstoreconnect.apple.com".
    -scmProvider                                             which implementation to use for Git operations (system/xcode)
    -showBuildTimingSummary                                  display a report of the timings of all the commands invoked during the build
    -create-xcframework                                      create an xcframework from prebuilt libraries; -help for more information.
    -extractSparseResultBundle                               specifies that a sparse result bundle should be extracted from the result bundle specified by -baseResultBundlePath and written to the location specified by -sparseResultBundlePath
    -baseResultBundlePath PATH                               specifies the path to the base result bundle
    -sparseResultBundlePath PATH                             specifies the path where the sparse result bundle will be written
    -extract-only BUNDLE_NAME/TEST_IDENTIFIER                specifies the test for which results should be included in the sparse result bundle
    -needsUpgrade                                            determine whether there are any project modernizations or Swift migrations to perform
    -ssh-auth-sock-env VALUE                                 value of SSH_AUTH_SOCK environment variable
    -ssh-agent-pid-env VALUE                                 value of SSH_AGENT_PID environment variable
    -git-ssh-command-env VALUE                               value of GIT_SSH_COMMAND environment variable
make[1]: *** [release] Error 64
make: *** [release] Error 2
$

[David Kilzer (:ddkilzer)]

Comment on attachment 435566 [details]
Patch v1

Going to cq+ this for now; if there are additional concerns, we can always revert this change.

[EWS]

Committed r281082 (240542@main): <https://commits.webkit.org/240542@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 435566 [details].

[Radar WebKit Bug Importer]

<rdar://problem/81981663>

[Alexey Proskuryakov]

> The error message seems pretty explicit about it:

This is not what I was asking about. I was asking about including sanitizer.xcconfig twice. This happens because it's included by both asan.xcconfig and ubsan.xcconfig. There is a good chance that some settings are messed up (at least duplicated) because of this.

[David Kilzer (:ddkilzer)]

(In reply to Alexey Proskuryakov from comment #9)
> > The error message seems pretty explicit about it:
> 
> This is not what I was asking about. I was asking about including
> sanitizer.xcconfig twice. This happens because it's included by both
> asan.xcconfig and ubsan.xcconfig. There is a good chance that some settings
> are messed up (at least duplicated) because of this.

Bug 230100: Fix ASan+UBSan builds: part two
<https://bugs.webkit.org/show_bug.cgi?id=230100>

The issue Alexey described was not a problem, but I found other issues while investigating that.