Bug 232000 – Apple-Win-10-Release-Build failing due to vc142.pdb write lock

• Status: CLOSED
• Resolution: CONFIGURATION CHANGED
• Component: New Bugs

[Jed Spraul]

Apple-Win-10-Release-Build failing due to vc142.pdb write lock

[Jed Spraul]

Created attachment 441845 [details]
Patch

[Jed Spraul]

Apple-Win-10-Release-Build has failed for 8+ days with an error similar to the following (seeming on random .cpp files, this from the first failed build of ~350 so far - build 6109):

fatal error C1041: cannot open program database 'C:\cygwin\worker\win10-release\build\WebKitBuild\Release\Tools\TestWebKitAPI\TestWebCoreLib.dir\Release\vc142.pdb'; if multiple CL.EXE write to the same .PDB file, please use /FS [C:\cygwin\worker\win10-release\build\WebKitBuild\Release\Tools\TestWebKitAPI\TestWebCoreLib.vcxproj]

The output doesn't appear to be categorized by the buildbot as an error.

As far as a patch, I added the /FS option to the same line turning on the PDB generation. Even the documentation states this might not solve the problem, but it could be the simplest fix. In any case, it may be hard to recreate what's happening on buildbot on a developer's machine.

I only spot-checked a handful of the failed builds, but it seemed like this was primarily an issue in the TestWebCoreLib project, so a fix may only be necessary for that one project rather than across all projects. In addition, the Apple-Win-10-Debug-Build build hasn't been affected, so a Release-only tweak may be enough.

It wasn't clear why the error began occurring either, perhaps some configuration change on the buildbot.

Research:

 https://stackoverflow.com/questions/20643370/visual-studio-2013-fatal-error-c1041-fs

  - the proper solution is per-project intermediate file output directories
  - file locks may be caused by other software, such as antivirus/backup
  - fix had to be applied to all projects for several
  - 1 claim that /FS must also be passed to the linker, doesn't make sense
  - could disable parallel build (slow) or enable in-obj symbols (large)
  - the MSPDBSRV.EXE sync process used by /FS can get wedged
  - some file(s) may become corrupted (especially PDB?), requiring deleting build files, not an issue on a clean build

 https://docs.microsoft.com/en-us/cpp/build/reference/fs-force-synchronous-pdb-writes

  - /FS syncs PDB writes through MSPDBSRV.EXE, longer builds + doesn't prevent all errors
  - recommend separate intermediate and output locations for all projects
  - /MP (not-Ninja per OptionsMSVC.cmake) enables /FS

[Fujii Hironori]

We had a discussion in WebKit Slack today.
https://webkit.slack.com/archives/CU64U6FDW/p1634674617427300
Because AppleWin EWS doesn't have this issue, we should try clean build at first.

[Fujii Hironori]

The AppleWin bot (bot548) got back to normal.
https://build.webkit.org/#/builders/67/builds/6460

Ryan Haddad said:
> Looks like my manual intervention worked.

[Fujii Hironori]

Comment on attachment 441845 [details]
Patch

I think we no longer need this patch.

[Jed Spraul]

👍