Bug 77327 – [CMAKE] Fix build break because of memory exhausted.

• Status: RESOLVED
• Resolution: FIXED
• Component: WebKit EFL

[Ryuan Choi]

Linking is failed in debug build.

I used "./Tools/Scripts/build-webkit --efl --debug" and get below error.

Linking CXX executable ../../../Programs/DumpRenderTree
/usr/bin/ld: final link failed: Memory exhausted
collect2: ld returned 1 exit status
make[2]: *** [Programs/DumpRenderTree] 오류 1
make[1]: *** [Tools/DumpRenderTree/efl/CMakeFiles/Programs/DumpRenderTree.dir/all] 오류 2
make: *** [all] 오류 2

I think that we need some tricks like Qt port did.

[Ryuan Choi]

Created attachment 124691 [details]
Patch

[WebKit Review Bot]

Attachment 124691 [details] did not pass style-queue:

Failed to run "['Tools/Scripts/update-webkit']" exit_code: 9

Updating OpenSource
First, rewinding head to replay your work on top of it...
Applying: Fix compilation errors on build-webkit --debug --no-workers on mac.
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
Auto-merging LayoutTests/ChangeLog
CONFLICT (content): Merge conflict in LayoutTests/ChangeLog
Auto-merging LayoutTests/platform/qt/Skipped
CONFLICT (content): Merge conflict in LayoutTests/platform/qt/Skipped
Auto-merging Source/WebCore/ChangeLog
CONFLICT (content): Merge conflict in Source/WebCore/ChangeLog
Failed to merge in the changes.
Patch failed at 0001 Fix compilation errors on build-webkit --debug --no-workers on mac.

When you have resolved this problem run "git rebase --continue".
If you would prefer to skip this patch, instead run "git rebase --skip".
To restore the original branch and stop rebasing run "git rebase --abort".

rebase refs/remotes/origin/master: command returned error: 1

Died at Tools/Scripts/update-webkit line 164.


If any of these errors are false positives, please file a bug against check-webkit-style.

[Ryuan Choi]

Created attachment 124692 [details]
Patch

[WebKit Review Bot]

Attachment 124692 [details] did not pass style-queue:

Failed to run "['Tools/Scripts/update-webkit']" exit_code: 9

Updating OpenSource
From git://git.webkit.org/WebKit
   3b4a9fa..2a654fd  master     -> origin/master
Partial-rebuilding .git/svn/refs/remotes/origin/master/.rev_map.268f45cc-cd09-0410-ab3c-d52691b4dbfc ...
Currently at 106348 = 3b4a9fa133678770a77bc6d602361e3cd4cb2dd1
r106349 = 976b1826441f3aef8b3a6be4636b2bbcb98365cc
r106350 = 2a654fd8513245efac73e1f8526121f158404c41
Done rebuilding .git/svn/refs/remotes/origin/master/.rev_map.268f45cc-cd09-0410-ab3c-d52691b4dbfc
First, rewinding head to replay your work on top of it...
Applying: Fix compilation errors on build-webkit --debug --no-workers on mac.
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
Auto-merging LayoutTests/ChangeLog
CONFLICT (content): Merge conflict in LayoutTests/ChangeLog
Auto-merging LayoutTests/platform/qt/Skipped
CONFLICT (content): Merge conflict in LayoutTests/platform/qt/Skipped
Auto-merging Source/WebCore/ChangeLog
CONFLICT (content): Merge conflict in Source/WebCore/ChangeLog
Failed to merge in the changes.
Patch failed at 0001 Fix compilation errors on build-webkit --debug --no-workers on mac.

When you have resolved this problem run "git rebase --continue".
If you would prefer to skip this patch, instead run "git rebase --skip".
To restore the original branch and stop rebasing run "git rebase --abort".

rebase refs/remotes/origin/master: command returned error: 1

Died at Tools/Scripts/update-webkit line 164.


If any of these errors are false positives, please file a bug against check-webkit-style.

[Gyuyoung Kim]

Comment on attachment 124692 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=124692&action=review

> Source/WebCore/ChangeLog:8
> +        Use SVGAllInOne files to avoid memory exhaustion for debug builds on linux

I don't understand well what do you mean in this changelog.

[Ryuan Choi]

Created attachment 124700 [details]
Patch

[WebKit Review Bot]

Attachment 124700 [details] did not pass style-queue:

Failed to run "['Tools/Scripts/update-webkit']" exit_code: 9

Updating OpenSource
First, rewinding head to replay your work on top of it...
Applying: Fix compilation errors on build-webkit --debug --no-workers on mac.
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
Auto-merging LayoutTests/ChangeLog
CONFLICT (content): Merge conflict in LayoutTests/ChangeLog
Auto-merging LayoutTests/platform/qt/Skipped
CONFLICT (content): Merge conflict in LayoutTests/platform/qt/Skipped
Auto-merging Source/WebCore/ChangeLog
CONFLICT (content): Merge conflict in Source/WebCore/ChangeLog
Failed to merge in the changes.
Patch failed at 0001 Fix compilation errors on build-webkit --debug --no-workers on mac.

When you have resolved this problem run "git rebase --continue".
If you would prefer to skip this patch, instead run "git rebase --skip".
To restore the original branch and stop rebasing run "git rebase --abort".

rebase refs/remotes/origin/master: command returned error: 1

Died at Tools/Scripts/update-webkit line 164.


If any of these errors are false positives, please file a bug against check-webkit-style.

[Ryuan Choi]

(In reply to comment #5)
> (From update of attachment 124692 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=124692&action=review
> 
> > Source/WebCore/ChangeLog:8
> > +        Use SVGAllInOne files to avoid memory exhaustion for debug builds on linux
> 
> I don't understand well what do you mean in this changelog.

little bit changed.

BTW, I don't know why style-queue failed to merge. :(

[Ryuan Choi]

Created attachment 125751 [details]
Patch

[Ryuan Choi]

(In reply to comment #9)
> Created an attachment (id=125751) [details]
> Patch

rebased for green style bot.

[Raphael Kubo da Costa (:rakuco)]

It looks like a very hackish solution tuned to work in a very specific environment. The right solution would be to just get more RAM :)

Building webkit in debug mode is a very heavy task, but it still works here in a 4GB machine.

[Antonio Gomes]

guys, we should keep three eyes at it:)

[Raphael Kubo da Costa (:rakuco)]

Comment on attachment 125751 [details]
Patch

r-'ing the currently proposed patch.

[Ryuan Choi]

(In reply to comment #11)
> It looks like a very hackish solution tuned to work in a very specific environment. The right solution would be to just get more RAM :)
> 
> Building webkit in debug mode is a very heavy task, but it still works here in a 4GB machine.

OK, I still used this to debug, but I agree that this is little bit hackish.

Anyway, I also used 4GB RAM.

[Ryuan Choi]

Reopening to attach new patch.

[Ryuan Choi]

Created attachment 134016 [details]
anotherApproach

[Ryuan Choi]

(In reply to comment #16)
> Created an attachment (id=134016) [details]
> anotherApproach

I tried different approach after reading https://lists.webkit.org/pipermail/webkit-dev/2012-March/020111.html

[Raphael Kubo da Costa (:rakuco)]

Comment on attachment 134016 [details]
anotherApproach

View in context: https://bugs.webkit.org/attachment.cgi?id=134016&action=review

> Source/cmake/WebKitHelpers.cmake:61
> +    IF ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU")
> +        GET_TARGET_PROPERTY(OLD_LINK_FLAGS ${_target} LINK_FLAGS)
> +
> +        IF (WTF_CPU_X86)
> +            SET(OLD_LINK_FLAGS "${OLD_LINK_FLAGS} -Wl,--no-keep-memory")
> +        ENDIF ()
> +
> +        SET_TARGET_PROPERTIES (${_target} PROPERTIES
> +            LINK_FLAGS "${OLD_LINK_FLAGS}")
> +    ENDIF ()

You could rewrite this as
  IF ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU" AND WTF_CPU_X86)
      # Update target properties
  ENDIF ()

It would be good to add a comment explaining why this is being done, perhaps even pointing to the mentioned link.

[Raphael Kubo da Costa (:rakuco)]

BTW, are you still unable to link WebCore on your machine?

[Ryuan Choi]

Created attachment 134174 [details]
anotherApproach2

[Ryuan Choi]

(In reply to comment #18)
> (From update of attachment 134016 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=134016&action=review
> 
> > Source/cmake/WebKitHelpers.cmake:61
> > +    IF ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU")
> > +        GET_TARGET_PROPERTY(OLD_LINK_FLAGS ${_target} LINK_FLAGS)
> > +
> > +        IF (WTF_CPU_X86)
> > +            SET(OLD_LINK_FLAGS "${OLD_LINK_FLAGS} -Wl,--no-keep-memory")
> > +        ENDIF ()
> > +
> > +        SET_TARGET_PROPERTIES (${_target} PROPERTIES
> > +            LINK_FLAGS "${OLD_LINK_FLAGS}")
> > +    ENDIF ()
> 
> You could rewrite this as
>   IF ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU" AND WTF_CPU_X86)
>       # Update target properties
>   ENDIF ()
> 
Done. and checked 'DEBUG' also.

> It would be good to add a comment explaining why this is being done, perhaps even pointing to the mentioned link.
Add some comment.


(In reply to comment #19)
> BTW, are you still unable to link WebCore on your machine?

Yes, so I just disabled SVG when I want debug binary.
and also, my collegue, who test SVG feature, requests this patch.

[Raphael Kubo da Costa (:rakuco)]

Comment on attachment 134174 [details]
anotherApproach2

View in context: https://bugs.webkit.org/attachment.cgi?id=134174&action=review

No objections from my side besides the comment below.

> Source/cmake/WebKitHelpers.cmake:52
> +    IF ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU" AND WTF_CPU_X86 AND CMAKE_BUILD_TYPE STREQUAL Debug)

Please enclose Debug within quotes, just like "GNU".

[Ryuan Choi]

Created attachment 134183 [details]
anotherApproach3

[Ryuan Choi]

(In reply to comment #22)
> (From update of attachment 134174 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=134174&action=review
> 
> No objections from my side besides the comment below.
> 
> > Source/cmake/WebKitHelpers.cmake:52
> > +    IF ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU" AND WTF_CPU_X86 AND CMAKE_BUILD_TYPE STREQUAL Debug)
> 
> Please enclose Debug within quotes, just like "GNU".

Thanks, I tried like you mentioned.

[Ryuan Choi]

Created attachment 163543 [details]
Patch

[Raphael Kubo da Costa (:rakuco)]

Comment on attachment 163543 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=163543&action=review

> Source/cmake/WebKitHelpers.cmake:59
> +    IF ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU"
> +            AND WTF_CPU_X86
> +            AND "${CMAKE_BUILD_TYPE}" STREQUAL "Debug")

You could as well just have put this into a single line.

> Source/cmake/WebKitHelpers.cmake:66
> +        SET_TARGET_PROPERTIES (${_target} PROPERTIES
> +            LINK_FLAGS "${OLD_LINK_FLAGS}")

Ditto.

[Ryuan Choi]

Created attachment 163548 [details]
Patch

[Ryuan Choi]

Created attachment 163550 [details]
Patch

[Patrick R. Gansterer]

Comment on attachment 163550 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=163550&action=review

> Source/cmake/WebKitHelpers.cmake:63
> +        SET_TARGET_PROPERTIES (${_target} PROPERTIES LINK_FLAGS "${OLD_LINK_FLAGS}")

do not set the linker flags on the target. use the CMAKE_XXX variables instead. If it's not possible then write the reason in the ChangeLog.

[Raphael Kubo da Costa (:rakuco)]

Comment on attachment 163550 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=163550&action=review

>> Source/cmake/WebKitHelpers.cmake:63
>> +        SET_TARGET_PROPERTIES (${_target} PROPERTIES LINK_FLAGS "${OLD_LINK_FLAGS}")
> 
> do not set the linker flags on the target. use the CMAKE_XXX variables instead. If it's not possible then write the reason in the ChangeLog.

Why?

[Patrick R. Gansterer]

Comment on attachment 163550 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=163550&action=review

>>> Source/cmake/WebKitHelpers.cmake:63
>>> +        SET_TARGET_PROPERTIES (${_target} PROPERTIES LINK_FLAGS "${OLD_LINK_FLAGS}")
>> 
>> do not set the linker flags on the target. use the CMAKE_XXX variables instead. If it's not possible then write the reason in the ChangeLog.
> 
> Why?

beause linker flags (or at least the the lags in this patch) are sth project-wide and calling WEBKIT_SET_EXTRA_LINKER_FLAGS() for every target is as ugly as the WEBKIT_SET_EXTRA_COMPILER_FLAGS() calls are already

[Raphael Kubo da Costa (:rakuco)]

Comment on attachment 163550 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=163550&action=review

>>>> Source/cmake/WebKitHelpers.cmake:63
>>>> +        SET_TARGET_PROPERTIES (${_target} PROPERTIES LINK_FLAGS "${OLD_LINK_FLAGS}")
>>> 
>>> do not set the linker flags on the target. use the CMAKE_XXX variables instead. If it's not possible then write the reason in the ChangeLog.
>> 
>> Why?
> 
> beause linker flags (or at least the the lags in this patch) are sth project-wide and calling WEBKIT_SET_EXTRA_LINKER_FLAGS() for every target is as ugly as the WEBKIT_SET_EXTRA_COMPILER_FLAGS() calls are already

I don't really see a problem with setting these properties per target. In this case it wouldn't make much difference anyway. I'm OK with setting this in a project-wide way as well.

[Ryuan Choi]

Created attachment 163575 [details]
Patch

[Ryuan Choi]

(In reply to comment #29)
> (From update of attachment 163550 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=163550&action=review
> 
> > Source/cmake/WebKitHelpers.cmake:63
> > +        SET_TARGET_PROPERTIES (${_target} PROPERTIES LINK_FLAGS "${OLD_LINK_FLAGS}")
> 
> do not set the linker flags on the target. use the CMAKE_XXX variables instead. If it's not possible then write the reason in the ChangeLog.

Sorry for late answer. I am compiling. :)

I tried to use CMAKE_SHARED_LINKER_FLAGS.
I hope that it is what you mentioned.


Thank you.

[Patrick R. Gansterer]

Comment on attachment 163575 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=163575&action=review

> I hope that it is what you mentioned.

Yes, thanks!

> Source/cmake/OptionsCommon.cmake:29
> +IF ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU" AND WTF_CPU_X86 AND "${CMAKE_BUILD_TYPE}" STREQUAL "Debug")

is WTF_CPU_X86 required? imho the WTF_CPU_X86 means the target platform and not the compiler platform. Is there a reason why you add it only to the Debug build? Are there problems with this flags in Release builds?

[Ryuan Choi]

(In reply to comment #35)
> (From update of attachment 163575 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=163575&action=review
> 
> > I hope that it is what you mentioned.
> 
> Yes, thanks!
> 
> > Source/cmake/OptionsCommon.cmake:29
> > +IF ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU" AND WTF_CPU_X86 AND "${CMAKE_BUILD_TYPE}" STREQUAL "Debug")
> 
> is WTF_CPU_X86 required? imho the WTF_CPU_X86 means the target platform and not the compiler platform. Is there a reason why you add it only to the Debug build? Are there problems with this flags in Release builds?

Oops, I did not catch it.
If it is not for compiler platform. I think that I should fix it.

And this option has trade off between speed and memory.
So I added it to debug build.

[Ryuan Choi]

Created attachment 163583 [details]
Patch

[Ryuan Choi]

(In reply to comment #35)
> (From update of attachment 163575 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=163575&action=review
> 
> > I hope that it is what you mentioned.
> 
> Yes, thanks!
> 
> > Source/cmake/OptionsCommon.cmake:29
> > +IF ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU" AND WTF_CPU_X86 AND "${CMAKE_BUILD_TYPE}" STREQUAL "Debug")
> 
> is WTF_CPU_X86 required? imho the WTF_CPU_X86 means the target platform and not the compiler platform. Is there a reason why you add it only to the Debug build? Are there problems with this flags in Release builds?

Changed to use CMAKE_HOST_SYSTEM_PROCESSOR instead of CMAKE_SYSTEM_PROCESSOR

Thank you.

[Ryuan Choi]

Created attachment 164501 [details]
Patch

[Gyuyoung Kim]

Comment on attachment 164501 [details]
Patch

Looks fine. Patrick also agreed this patch.

[WebKit Review Bot]

Comment on attachment 164501 [details]
Patch

Clearing flags on attachment: 164501

Committed r128855: <http://trac.webkit.org/changeset/128855>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.