232000 2021-10-19 20:45:48 -0700 Apple-Win-10-Release-Build failing due to vc142.pdb write lock 2021-10-20 04:55:13 -0700 1 1 1 Unclassified WebKit New Bugs WebKit Nightly Build Unspecified Unspecified CLOSED CONFIGURATION CHANGED P2 Normal --- 1 jspraul webkit-unassigned annulen ews-watchlist fujii gyuyoung.kim ryuan.choi sergio oldest_to_newest 1806477 0 jspraul 2021-10-19 20:45:48 -0700 Apple-Win-10-Release-Build failing due to vc142.pdb write lock 1806482 1 441845 jspraul 2021-10-19 21:00:10 -0700 Created attachment 441845 Patch 1806496 2 jspraul 2021-10-19 21:57:40 -0700 Apple-Win-10-Release-Build has failed for 8+ days with an error similar to the following (seeming on random .cpp files, this from the first failed build of ~350 so far - build 6109): fatal error C1041: cannot open program database 'C:\cygwin\worker\win10-release\build\WebKitBuild\Release\Tools\TestWebKitAPI\TestWebCoreLib.dir\Release\vc142.pdb'; if multiple CL.EXE write to the same .PDB file, please use /FS [C:\cygwin\worker\win10-release\build\WebKitBuild\Release\Tools\TestWebKitAPI\TestWebCoreLib.vcxproj] The output doesn't appear to be categorized by the buildbot as an error. As far as a patch, I added the /FS option to the same line turning on the PDB generation. Even the documentation states this might not solve the problem, but it could be the simplest fix. In any case, it may be hard to recreate what's happening on buildbot on a developer's machine. I only spot-checked a handful of the failed builds, but it seemed like this was primarily an issue in the TestWebCoreLib project, so a fix may only be necessary for that one project rather than across all projects. In addition, the Apple-Win-10-Debug-Build build hasn't been affected, so a Release-only tweak may be enough. It wasn't clear why the error began occurring either, perhaps some configuration change on the buildbot. Research: https://stackoverflow.com/questions/20643370/visual-studio-2013-fatal-error-c1041-fs - the proper solution is per-project intermediate file output directories - file locks may be caused by other software, such as antivirus/backup - fix had to be applied to all projects for several - 1 claim that /FS must also be passed to the linker, doesn't make sense - could disable parallel build (slow) or enable in-obj symbols (large) - the MSPDBSRV.EXE sync process used by /FS can get wedged - some file(s) may become corrupted (especially PDB?), requiring deleting build files, not an issue on a clean build https://docs.microsoft.com/en-us/cpp/build/reference/fs-force-synchronous-pdb-writes - /FS syncs PDB writes through MSPDBSRV.EXE, longer builds + doesn't prevent all errors - recommend separate intermediate and output locations for all projects - /MP (not-Ninja per OptionsMSVC.cmake) enables /FS 1806505 3 fujii 2021-10-19 23:01:09 -0700 We had a discussion in WebKit Slack today. https://webkit.slack.com/archives/CU64U6FDW/p1634674617427300 Because AppleWin EWS doesn't have this issue, we should try clean build at first. 1806521 4 fujii 2021-10-19 23:43:56 -0700 The AppleWin bot (bot548) got back to normal. https://build.webkit.org/#/builders/67/builds/6460 Ryan Haddad said: > Looks like my manual intervention worked. 1806524 5 441845 fujii 2021-10-20 00:06:55 -0700 Comment on attachment 441845 Patch I think we no longer need this patch. 1806571 6 jspraul 2021-10-20 04:55:13 -0700 👍 441845 2021-10-19 21:00:10 -0700 2021-10-20 00:06:55 -0700 Patch bug-232000-20211020000010.patch text/plain 1347 jspraul U3VidmVyc2lvbiBSZXZpc2lvbjogMjg0NTE2CmRpZmYgLS1naXQgYS9Tb3VyY2UvY21ha2UvT3B0 aW9uc01TVkMuY21ha2UgYi9Tb3VyY2UvY21ha2UvT3B0aW9uc01TVkMuY21ha2UKaW5kZXggNWUx ZGYxOTc2Mjk3MjY1MWMxNTgxYTQ3ZDdkMjE2Y2FjNjk4YjU3Yi4uMGNkNWQ0NTNiNTQ4YmIzMTll Y2EzNzVhZDdjNTZkYjBiOWJjY2EyNSAxMDA2NDQKLS0tIGEvU291cmNlL2NtYWtlL09wdGlvbnNN U1ZDLmNtYWtlCisrKyBiL1NvdXJjZS9jbWFrZS9PcHRpb25zTVNWQy5jbWFrZQpAQCAtMTA2LDcg KzEwNiw3IEBAIGFkZF9jb21waWxlX29wdGlvbnMoCiApCiAKICMgQ3JlYXRlIHBkYiBmaWxlcyBm b3IgZGVidWdnaW5nIHB1cnBvc2VzLCBhbHNvIGZvciBSZWxlYXNlIGJ1aWxkcwotYWRkX2NvbXBp bGVfb3B0aW9ucygvWmkgL0dTKQorYWRkX2NvbXBpbGVfb3B0aW9ucygvWmkgL0dTIC9GUykKIAog c2V0KENNQUtFX1NIQVJFRF9MSU5LRVJfRkxBR1MgIiR7Q01BS0VfU0hBUkVEX0xJTktFUl9GTEFH U30gL0RFQlVHIC9PUFQ6SUNGIC9PUFQ6UkVGIikKIHNldChDTUFLRV9FWEVfTElOS0VSX0ZMQUdT ICIke0NNQUtFX0VYRV9MSU5LRVJfRkxBR1N9IC9ERUJVRyAvT1BUOklDRiAvT1BUOlJFRiIpCmRp ZmYgLS1naXQgYS9DaGFuZ2VMb2cgYi9DaGFuZ2VMb2cKaW5kZXggOGMwYmRlM2IyODRjZjY1NjBi ZTIxY2E2NTJlZjA5ODg4MGQwNjViZS4uYmE5MDE4ZTVkMjY3OWY0MmJhNzBhZmFhMDg4ZjMwNjFh Mzc3MzIzZiAxMDA2NDQKLS0tIGEvQ2hhbmdlTG9nCisrKyBiL0NoYW5nZUxvZwpAQCAtMSwzICsx LDE0IEBACisyMDIxLTEwLTE5ICBKZWQgU3ByYXVsICA8anNwcmF1bEB5YWhvby5jb20+CisKKyAg ICAgICAgQXBwbGUtV2luLTEwLVJlbGVhc2UtQnVpbGQgZmFpbGluZyBkdWUgdG8gdmMxNDIucGRi IHdyaXRlIGxvY2sKKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dp P2lkPTIzMjAwMAorCisgICAgICAgIFJldmlld2VkIGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAg ICAgICogU291cmNlL2NtYWtlL09wdGlvbnNNU1ZDLmNtYWtlOgorICAgICAgICBBZGQgL0ZTIG9w dGlvbiB0byBzZXJpYWxpemUgUERCIGFjY2VzcyBhcyByZWNvbW1lbmRlZCBpbiBidWlsZCBlcnJv ciBtZXNzYWdlLgorICAgICAgICAiVGhpcyBtYXkgbWFrZSBidWlsZHMgc2lnbmlmaWNhbnRseSBs b25nZXIsIGFuZCBpdCBkb2Vzbid0IHByZXZlbnQgYWxsIGVycm9ycyIKKwogMjAyMS0xMC0xOSAg Um9zcyBLaXJzbGluZyAgPHJvc3Mua2lyc2xpbmdAc29ueS5jb20+CiAKICAgICAgICAgW0NNYWtl XSBBbGlnbiBPcHRpb25zTWFjIHdpdGggWGNvZGUgYnVpbGQK