Bug 316000 – REGRESSION(314042@main): Intermittent build failures on CMake+Gtk/Swift builds

• Status: RESOLVED
• Resolution: FIXED
• Component: CMake

[Nikolas Zimmermann]

On the GTK EWS builder GTK-GTK3-LibWebRTC-Build-EWS (#173), build-webkit fails sporadically (passes on retry) with:

<snippet>
[267/2053] Generating ../../WebKitGTK/DerivedSources/InspectorGResourceBundle.c, ../../WebKitGTK/DerivedSources/InspectorGResourceBundle.c.deps
[269/2053] Generating ../../DerivedSources/WebKit/GeneratedSerializers.h, ../../DerivedSources/WebKit/GeneratedSerializersShared.cpp, ../../DerivedSources/WebKit/GeneratedSerializersWebProcess.cpp, ../../DerivedSources/WebKit/GeneratedSerializersGPUProcess.cpp, ../../DerivedSources/WebKit/GeneratedSerializersNetworkProcess.cpp, ../../DerivedSources/WebKit/GeneratedSerializersPlatform.cpp, ../../DerivedSources/WebKit/GeneratedSerializersModelProcess.cpp, ../../DerivedSources/WebKit/GeneratedSerializersUIProcess.cpp, ../../DerivedSources/WebKit/GeneratedSerializersCommon.cpp, ../../DerivedSources/WebKit/GeneratedWebKitSecureCoding.h, ../../DerivedSources/WebKit/GeneratedWebKitSecureCoding.cpp, ../../DerivedSources/WebKit/SerializedTypeInfo.cpp, ../../DerivedSources/WebKit/WebKitPlatformGeneratedSerializers.cpp
generate-serializers.py: 6 type(s) fell into GeneratedSerializersCommon.cpp (residual bucket): WebCore::DocumentSyncData, WebCore::DocumentSyncSerializationData, WebCore::FrameTreeSyncData, WebCore::FrameTreeSyncSerializationData, WebCore::SharedTimebaseHandle, WebKit::SharedPreferencesForWebProcess
[270/2053] Generating WebKit Swift-C++ header
FAILED: DerivedSources/WebKit/WebKit-Swift-CPP.h.stamp DerivedSources/WebKit/WebKit-Swift-CPP.h /sdk/webkit/WebKitBuild/GTK/Release/DerivedSources/WebKit/WebKit-Swift-CPP.h.stamp /sdk/webkit/WebKitBuild/GTK/Release/DerivedSources/WebKit/WebKit-Swift-CPP.h 
cd /sdk/webkit/WebKitBuild/GTK/Release/Source/WebKit && /usr/bin/cmake -E copy_if_different /sdk/webkit/WebKitBuild/GTK/Release/DerivedSources/WebKit/WebKit-Swift-CPP.h.tmp /sdk/webkit/WebKitBuild/GTK/Release/DerivedSources/WebKit/WebKit-Swift-CPP.h && /usr/bin/cmake -E touch /sdk/webkit/WebKitBuild/GTK/Release/DerivedSources/WebKit/WebKit-Swift-CPP.h.stamp
Error copying file (if different) from "/sdk/webkit/WebKitBuild/GTK/Release/DerivedSources/WebKit/WebKit-Swift-CPP.h.tmp" to "/sdk/webkit/WebKitBuild/GTK/Release/DerivedSources/WebKit/WebKit-Swift-CPP.h".
[271/2053] Generating ../../JavaScriptCore/DerivedSources/LLIntDesiredSettings.h
ninja: build stopped: subcommand failed.
</snippet>

See https://ews-build.webkit.org/#/builders/173/builds/22800 for the full log.
Same happend on e.g. builds 22806, 22804. The without-change rebuild then succeeds, so EWS blames the PR, but it is environmental flakiness, not the change.

[Nikolas Zimmermann]

Before 314042@main a single, independent `swiftc -typecheck` command was the _only_ writer of WebKit-Swift-CPP.h.tmp, and it copied in the same step -> one writer, no race.
314042@main removed that command, made the header a side-effect of the real compile, and because the legacy (pre-CMP0157 -- we still have CMake 3.28 in the CI) model compiles Swift in the link step - added the WebKit_SwiftCompile helper. The helper inherits the emit flag, so now both the helper and the real libwebkitgtk link write the same temp, and the copy waits on only one....


From build.ninja

1. First writer

```
#############################################
# Link the shared library lib/libwebkitgtk-6.0.so.4.17.2

build lib/libwebkitgtk-6.0.so.4.17.2 Source/WebKit/CMakeFiles/WebKit.dir/UIProcess/SwiftDemoLogo.swift.o ... -emit-clang-header-path /sdk/webkit/WebKitBuild/GTK/Release/DerivedSources/WebKit/WebKit-Swift-CPP.h.tmp ...
  INCLUDES = -I /sdk/webkit/WebKitBuild/GTK/Release/Source/WebKit/include ...
...
```

2. Second writer

```
#############################################
# Link the static library lib/libWebKit_SwiftCompile.a
  
build lib/libWebKit_SwiftCompile.a Source/WebKit/CMakeFiles/WebKit_SwiftCompile.dir/UIProcess/SwiftDemoLogo.swift.o  -emit-clang-header-path /sdk/webkit/WebKitBuild/GTK/Release/DerivedSources/WebKit/WebKit-Swift-CPP.h.tmp ...
  INCLUDES = -I /opt/swift/usr/lib/swift -I /sdk/webkit/WebKitBuild/GTK/Release/Source/WebKit/include ...
...
```

--> Two  rules write the same WebKit-Swift-CPP.h.tmp; the copy waits for one of them and reads the file while the other may still be writing it. Since neither rule declares .tmp as a ninja output,  ninja never serializes them. That's my guess for the sporadic error "Error copying file" that we observe...

We need to support CMake < 3.29, so needs a proper fix. Newer CMake versions are not affected, since the -emit-clang-header-path... is only done once.

[Nikolas Zimmermann]

Pull request: https://github.com/WebKit/WebKit/pull/66199

[EWS]

Committed 314322@main (9584faf20f98): <https://commits.webkit.org/314322@main>

Reviewed commits have been landed. Closing PR #66199 and removing active labels.