Bug 155872

Summary: [Win] CMake seems to build all generated files every time
Product: WebKit Reporter: Brent Fulgham <bfulgham>
Component: WebKit Misc.Assignee: Brent Fulgham <bfulgham>
Status: RESOLVED DUPLICATE    
Severity: Normal CC: achristensen, bfulgham, commit-queue, don.olmstead, Hironori.Fujii, peavo
Priority: P2    
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=154938
Bug Depends on: 155986    
Bug Blocks:    
Attachments:
Description Flags
Patch
none
Patch
none
Patch achristensen: review+

Description Brent Fulgham 2016-03-24 22:32:48 PDT
It seems like the CMake-based build causes the IDL files to be processed in full every build. This is super annoying, especially since this has become a single-threaded bottleneck during the build.
Comment 1 Brent Fulgham 2016-03-24 22:42:43 PDT
The bottleneck seems to be that Visual Studio waits for each invocation of the "Custom Build Step" to complete:

setlocal
cd C:\Projects\WebKit\OpenSource\Source\WebCore
if %errorlevel% neq 0 goto :cmEnd
C:
if %errorlevel% neq 0 goto :cmEnd
C:\cygwin\bin\perl.exe -IC:/Projects/WebKit/OpenSource/Source/WebCore/bindings/scripts C:/Projects/WebKit/OpenSource/Source/WebCore/bindings/scripts/generate-bindings.pl --defines "LANGUAGE_JAVASCRIPT=1 " ... endless stuff

if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
Comment 2 Brent Fulgham 2016-03-24 22:43:41 PDT
I wonder what would happen if we did a "CALL" on the Perl program to run in a new process. We would lose the ability to end the build on an error in the script generation, but that doesn't really happen much anymore.
Comment 3 Brent Fulgham 2016-03-24 22:52:58 PDT
We probably need some way to sync that the auto-generating is done before compiling stuff that depends on it.
Comment 4 Don Olmstead 2016-03-25 10:18:50 PDT
One thought was maybe derived sources should be split into their own projects. https://bugs.webkit.org/show_bug.cgi?id=151399 had some initial work on this but didn't land.

And here's a related bug you might want to roll into this https://bugs.webkit.org/show_bug.cgi?id=154938 that I opened.
Comment 5 Brent Fulgham 2016-03-25 14:14:02 PDT
Created attachment 274940 [details]
Patch
Comment 6 Brent Fulgham 2016-03-25 14:53:03 PDT
Blah. Well, that didn't work.
Comment 7 Alexey Proskuryakov 2016-03-25 15:05:06 PDT
Comment on attachment 274940 [details]
Patch

CMake Error at Source/WebCore/CMakeLists.txt:3832 (add_library):
  Cannot find source file:

    C:/cygwin/home/buildbot/WebKit/WebKitBuild/Release/DerivedSources/WebCore/JSInternalSettings.cpp
Comment 8 Brent Fulgham 2016-03-25 16:26:20 PDT
Created attachment 274953 [details]
Patch
Comment 9 Brent Fulgham 2016-03-25 17:40:48 PDT
Created attachment 274961 [details]
Patch
Comment 10 Alex Christensen 2016-03-28 09:47:57 PDT
Comment on attachment 274961 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=274961&action=review

> Source/cmake/WebKitMacros.cmake:96
> +        set(_custom_outputs "${_destination}/${_prefix}${_name}.${_extension}" "${_destination}/${_prefix}${_name}.h")
> +        if (MSVC)
> +            set(_custom_outputs "${_destination}/${_prefix}${_name}.h")
> +        endif ()

if else endif
Comment 11 Brent Fulgham 2016-03-29 08:42:43 PDT
Committed r198781: <http://trac.webkit.org/changeset/198781>
Comment 12 WebKit Commit Bot 2016-03-29 13:34:42 PDT
Re-opened since this is blocked by bug 155986
Comment 13 Fujii Hironori 2016-10-06 23:28:09 PDT
My patch in Bug 161433 Comment 9 mitigates this problem because:

1) It solves the problem editting one IDL triggers regenerating all bindings
2) It runs generate-bindings.pl parallel

But, not perfectly fixed. I'll fix this issue after finishing Bug 161433.
Comment 14 Fujii Hironori 2016-10-30 19:10:19 PDT
I solved this problem in Bug 163774.

*** This bug has been marked as a duplicate of bug 163774 ***