Bug 187725 – [CMake][Ninja][Win] midl.exe fails to create, open and write a temporary file in parallel build in Docker container

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Fujii Hironori]

[WinCairo][EWS] midl : command line error MIDL1016 : cannot create intermediate file C:\Users\ContainerAdministrator\AppData\Local\Temp\MIDEFA7.tmp

https://webkit-queues.webkit.org/patch/345018/wincairo-ews

> midl : command line error MIDL1016 : cannot create intermediate file C:\Users\ContainerAdministrator\AppData\Local\Temp\MIDEFA7.tmp

[Fujii Hironori]

A following error looks similar:

https://webkit-queues.webkit.org/patch/343701/wincairo-ews

> FAILED: DerivedSources/WebKitLegacy/Interfaces/AccessibleStates.h 
> 
> cmd.exe /C "cd /D C:\WebKit-EWS\WebKit\WebKitBuild\Release\Source\WebKitLegacy && midl.exe /I C:/WebKit-EWS/WebKit/Source/WebKitLegacy/win/Interfaces /I C:/WebKit-EWS/WebKit/Source/WebKitLegacy/win/Interfaces/Accessible2 /I C:/WebKit-EWS/WebKit/WebKitBuild/Release/DerivedSources/WebKitLegacy/include /I C:/WebKit-EWS/WebKit/Source/WebKitLegacy/win /WX /char signed /env win32 /tlb C:/WebKit-EWS/WebKit/WebKitBuild/Release/bin64/AccessibleStates.tlb /out C:/WebKit-EWS/WebKit/WebKitBuild/Release/DerivedSources/WebKitLegacy/Interfaces /h C:/WebKit-EWS/WebKit/WebKitBuild/Release/DerivedSources/WebKitLegacy/Interfaces/AccessibleStates.h /iid AccessibleStates_i.c "/D \"__PRODUCTION__=01\"" C:/WebKit-EWS/WebKit/Source/WebKitLegacy/win/Interfaces/Accessible2/AccessibleStates.idl"
> 
> Microsoft (R) 32b/64b MIDL Compiler Version 8.01.0622 
> 
> Copyright (c) Microsoft Corporation. All rights reserved.
> 
> midl : error MIDL2212 : error while writing to file C:\Users\ContainerAdministrator\AppData\Local\Temp\MID9469.tmp

[Ross Kirsling]

Another one here. :(
https://webkit-queues.webkit.org/patch/348745/wincairo-ews

[Ross Kirsling]

Although the one in my previous comment was the same as the first (MIDL1016), https://webkit-queues.webkit.org/results/9093311 has yet a third error code:

> [4501/6255] Generating ../../DerivedSources/WebKitLegacy/Interfaces/AccessibleStates.h
> FAILED: DerivedSources/WebKitLegacy/Interfaces/AccessibleStates.h 
> cmd.exe /C "cd /D C:\WebKit-EWS\WebKit\WebKitBuild\Release\Source\WebKitLegacy && midl.exe /I C:/WebKit-EWS/WebKit/Source/WebKitLegacy/win/Interfaces /I C:/WebKit-EWS/WebKit/Source/WebKitLegacy/win/Interfaces/Accessible2 /I C:/WebKit-EWS/WebKit/WebKitBuild/Release/DerivedSources/WebKitLegacy/include /I C:/WebKit-EWS/WebKit/Source/WebKitLegacy/win /WX /char signed /env win32 /tlb C:/WebKit-EWS/WebKit/WebKitBuild/Release/bin64/AccessibleStates.tlb /out C:/WebKit-EWS/WebKit/WebKitBuild/Release/DerivedSources/WebKitLegacy/Interfaces /h C:/WebKit-EWS/WebKit/WebKitBuild/Release/DerivedSources/WebKitLegacy/Interfaces/AccessibleStates.h /iid AccessibleStates_i.c "/D \"__PRODUCTION__=01\"" C:/WebKit-EWS/WebKit/Source/WebKitLegacy/win/Interfaces/Accessible2/AccessibleStates.idl"
> Microsoft (R) 32b/64b MIDL Compiler Version 8.01.0622 
> 
> Copyright (c) Microsoft Corporation. All rights reserved.
> 
> midl : error MIDL2419 : Unable to open intermediate compiler file : Permission denied  (C:\Users\ContainerAdministrator\AppData\Local\Temp\MID1C3B.tmp)

[Fujii Hironori]

I can reproduce this issue by using Docker container on my PC.

perl Tools\Scripts\build-webkit --wincairo
cd WebKitBuild\Release
del DerivedSources/WebKitLegacy/Interfaces/*
ninja

If I invoke ninja -j1, it seems that it'd not happen.

[Fujii Hironori]

Created attachment 351227 [details]
Patch to run midl.exe sequentially

I created a patch to run midl.exe sequentially, but this doesn't address the issue.
So strange.

[Fujii Hironori]

Created attachment 351228 [details]
Patch to run midl.exe sequentially

Wrong patch attached. This is the patch I wanted to attach.

[Fujii Hironori]

Created attachment 351230 [details]
Patch to run midl.exe sequentially

[Fujii Hironori]

I created a PowerShell script to run midl.exe parallely in Docker.
https://gist.github.com/fujii/5dabb82dbd62dd535be6a44d6815b37b

Then, I got following errors.

> midl : error MIDL2419 : Unable to open intermediate compiler file : Permission denied  (C:\Users\ContainerAdministrator\AppData\Local\Temp\MID9E47.tmp)

> midl : command line error MIDL1016 : cannot create intermediate file C:\Users\ContainerAdministrator\AppData\Local\Temp\MID2B18.tmp

> midl : command line error MIDL1016 : cannot create intermediate file C:\Users\ContainerAdministrator\AppData\Local\Temp\MID3E58.tmp

But, If I run it sequentially, no such errors happened.

[Fujii Hironori]

My conclusion is Docker seems to have a problem of parallel file system access.
WinCairo port should give up either Docker or parallel build.

[Fujii Hironori]

Comment on attachment 351230 [details]
Patch to run midl.exe sequentially

These my patches are completely wrong. JOB_POOL_COMPILE target property doesn't affect to custom commands.
I should try USE_TERMINAL of add_custom_command to run midl.exe sequentially.

[Fujii Hironori]

I've confirmed it still happens even with USES_TERMINAL. :-(

[Fujii Hironori]

(In reply to Fujii Hironori from comment #11)
> I've confirmed it still happens even with USES_TERMINAL. :-(

This is wrong. I tested again. USES_TERMINAL actually solves this issue.

[Fujii Hironori]

Created attachment 353076 [details]
Patch

[Don Olmstead]

Informal r+

This will add the command to the job pool https://cmake.org/cmake/help/v3.13/prop_gbl/JOB_POOLS.html#prop_gbl:JOB_POOLS

[WebKit Commit Bot]

Comment on attachment 353076 [details]
Patch

Clearing flags on attachment: 353076

Committed r237416: <https://trac.webkit.org/changeset/237416>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/45555931>