Bug 108330 - [CMake] Use thin archives if building on Linux for non-shared-core debug builds
Summary: [CMake] Use thin archives if building on Linux for non-shared-core debug builds
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Jussi Kukkonen (jku)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-01-30 05:17 PST by Jussi Kukkonen (jku)
Modified: 2014-03-05 17:22 PST (History)
13 users (show)

See Also:


Attachments
Patch (2.20 KB, patch)
2014-02-09 15:06 PST, Ryuan Choi
no flags Details | Formatted Diff | Diff
Patch (3.47 KB, patch)
2014-02-09 15:26 PST, Ryuan Choi
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jussi Kukkonen (jku) 2013-01-30 05:17:05 PST
I've been trying to package webkit-efl lately, and noticed that we could use "thin archives" to get around the problem of .a archives growing over 4GB by using "thin archives". When thin archives are used while building static webcore_efl, the size of the archive goes from >4GB to <100MB.

"""
gnu ar can optionally create a thin archive, which contains a symbol index and references to the original copies of the member files of the archives. Such an archive is useful for building libraries for use within a local build, where the relocatable objects are expected to remain available, and copying the contents of each object would only waste time and space. Thin archives are also flattened, so that adding one or more archives to a thin archive will add the elements of the nested archive individually. The paths to the elements of the archive are stored relative to the archive itself.
"""

It's totally possible to leave this up to the distribution builders, but I think we could make it default as well (when building without SHARED_CORE) since I don't see why someone would not want this -- webkit-gtk already does the same, see bug 107400.

In practice the change would mean redefining these: 
  CMAKE_CXX_ARCHIVE_CREATE="<CMAKE_AR> cruT <TARGET> <LINK_FLAGS> <OBJECTS>"
  CMAKE_CXX_ARCHIVE_APPEND="<CMAKE_AR> ruT  <TARGET> <LINK_FLAGS> <OBJECTS>"
The only additions to default commands are "uT", but it seems we have to redefine the whole thing for that.

I'm still testing this, but filing a bug to keep track of it. CCing Kubo as he promised to take a look and comment whether it looks sane to him.
Comment 1 Jussi Kukkonen (jku) 2013-01-30 07:09:51 PST
Just saw thiago (CCd) on #webkit-gtk:

tmpsantos:   CXXLD  libwebkitgtk-3.0.la
tmpsantos: libtool: link: warning: `-version-info/-version-number' is ignored for convenience libraries
tmpsantos: ar: `x' cannot be used on thin archives.

So there may still be some problems with this, at least in some cases.
Comment 2 Tobias Mueller 2013-03-08 16:31:08 PST
Running into this, too:

make  all-am
make[1]: Entering directory `/home/muelli/svn/gnome2/WebKit'
/usr/bin/mkdir -p ./.deps/DerivedSources
/bin/sh ./libtool  --tag=CXX   --mode=link g++ -fvisibility-inlines-hidden -fno-rtti -g3 -pipe -O2 -Wno-c++11-compat -Wno-c++11-extensions -O2 -version-info 18:1:18 -Wl,--version-script,./Source/autotools/symbols.filter  -L/opt/gnome2/lib64 -Wl,-O0 -Wl,--enable-new-dtags -Wl,--sort-common -Wl,--as-needed -o libwebkitgtk-3.0.la   Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-AcceleratedCompositingContextClutter.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-AcceleratedCompositingContextGL.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-AcceleratedCompositingContextCairo.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-AssertMatchingEnums.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-ChromeClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-ContextMenuClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-DeviceMotionClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-DeviceOrientationClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-DocumentLoaderGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-DragClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-DumpRenderTreeSupportGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-EditorClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-FrameLoaderClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-FrameNetworkingContextGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-GeolocationClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-GtkAdjustmentWatcher.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-InspectorClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-NavigatorContentUtilsClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-PlatformStrategiesGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-UserMediaClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-WebViewInputMethodFilter.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitapplicationcache.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitdownload.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkiterror.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitfavicondatabase.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitfilechooserrequest.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitgeolocationpolicydecision.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitglobals.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkithittestresult.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkiticondatabase.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitnetworkrequest.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitnetworkresponse.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitsecurityorigin.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitsoupauthdialog.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitspellchecker.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitspellcheckerenchant.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitversion.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitviewportattributes.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebbackforwardlist.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebdatabase.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebdatasource.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebframe.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebhistoryitem.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebinspector.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebnavigationaction.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebpolicydecision.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebresource.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebplugin.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebplugindatabase.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebsettings.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebview.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebwindowfeatures.lo DerivedSources/webkit/libwebkitgtk_3_0_la-webkitenumtypes.lo DerivedSources/webkit/libwebkitgtk_3_0_la-webkitmarshal.lo  -lpthread libWebCore.la libWebCorePlatform.la libWebCoreModules.la libWebCoreDOM.la libWebCoreGtk.la   libjavascriptcoregtk-3.0.la -lcairo       -L/opt/gnome2/lib64 -lcairo -lfontconfig -lfreetype -lharfbuzz   -L/opt/gnome2/lib64 -lgailutil-3 -lgtk-3 -lgdk-3 -latk-1.0 -lgio-2.0 -lpangocairo-1.0 -lgdk_pixbuf-2.0 -lcairo-gobject -lpango-1.0 -lcairo -lgobject-2.0 -lglib-2.0     -Wl,--export-dynamic -pthread -L/opt/gnome2/lib64 -lgmodule-2.0 -lgthread-2.0 -lrt -lgio-2.0 -lgobject-2.0 -lglib-2.0    -L/opt/gnome2/lib64 -lgtk-3 -lgdk-3 -latk-1.0 -lgio-2.0 -lpangocairo-1.0 -lgdk_pixbuf-2.0 -lcairo-gobject -lpango-1.0 -lcairo -lgobject-2.0 -lglib-2.0   -ljpeg -L/opt/gnome2/lib64 -lsecret-1 -lgio-2.0 -lgobject-2.0 -lglib-2.0   -L/opt/gnome2/lib64 -lsoup-2.4 -lgio-2.0 -lgobject-2.0 -lglib-2.0   -L/opt/gnome2/lib64 -lxml2    -lGL -ldl -L/opt/gnome2/lib64 -lpangoft2-1.0 -lpango-1.0 -lfreetype -lfontconfig -lgobject-2.0 -lglib-2.0   -lpng15   -lsqlite3   -licui18n -licuuc -licudata  -lwebp -lXcomposite   -lXdamage -lXfixes   -lXrender -lX11   -lXt -lX11      -lz   -lrt 
libtool: link: warning: `-version-info/-version-number' is ignored for convenience libraries
libtool: link: rm -fr  .libs/libwebkitgtk-3.0.lax
libtool: link: (cd .libs/libwebkitgtk-3.0.lax/libWebCore.a && ar x "/home/muelli/svn/gnome2/WebKit/./.libs/libWebCore.a")
ar: `x' cannot be used on thin archives.
make[1]: *** [libwebkitgtk-3.0.la] Error 1
make[1]: Target `all-am' not remade because of errors.
make[1]: Leaving directory `/home/muelli/svn/gnome2/WebKit'
make: *** [all] Error 2


[jhbuild] muelli@bigbox ~/svn/gnome2/WebKit [±:master?] $ head config.log 
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.

It was created by WebKitGTK configure 1.11.5, which was
generated by GNU Autoconf 2.68.  Invocation command line was

  $ ./configure --prefix /opt/gnome2 --libdir /opt/gnome2/lib64 --enable-introspection --disable-silent-rules --with-gstreamer=1.0 --disable-static --disable-scrollkeeper --disable-gtk-doc --disable-docs --enable-gtk-doc=no --enable-gtk-doc-html=no --enable-gtk-doc-pdf=no --disable-webkit1 --disable-spellcheck --disable-glx --disable-egl --disable-gles2 --disable-gamepad --disable-video --disable-mediastream --disable-xslt --disable-geolocation --disable-svg --disable-svg-fonts --disable-web-audio --disable-coverage --disable-webgl --disable-gtk-doc-html --disable-accelerated-compositing LDFLAGS=-L/opt/gnome2/lib64 -Wl,-O0 -Wl,--enable-new-dtags -Wl,--sort-common -Wl,--as-needed CXXFLAGS=-g3 -pipe -O2 PKG_CONFIG_PATH=/opt/gnome2/lib64/pkgconfig:/opt/gnome2/share/pkgconfig:/usr/share/pkgconfig:/usr/lib64/pkgconfig:/usr/lib/pkgconfig:/opt/gnome/lib/pkgconfig --no-create --no-recursion
Comment 3 Ryuan Choi 2014-02-06 16:15:50 PST
Will you apply it ?

I think that it can be good idea.
Comment 4 Ryuan Choi 2014-02-09 15:06:24 PST
Created attachment 223650 [details]
Patch
Comment 5 Sergio Correia (qrwteyrutiyoup) 2014-02-09 15:14:20 PST
Comment on attachment 223650 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=223650&action=review

> Source/cmake/OptionsEfl.cmake:120
> +if (${CMAKE_SYSTEM_NAME} STREQUAL "Linux")
> +    set(CMAKE_CXX_ARCHIVE_CREATE "<CMAKE_AR> cruT <TARGET> <LINK_FLAGS> <OBJECTS>")
> +    set(CMAKE_C_ARCHIVE_CREATE "<CMAKE_AR> cruT <TARGET> <LINK_FLAGS> <OBJECTS>")
> +    set(CMAKE_CXX_ARCHIVE_APPEND "<CMAKE_AR> ruT <TARGET> <LINK_FLAGS> <OBJECTS>")
> +    set(CMAKE_C_ARCHIVE_APPEND "<CMAKE_AR> ruT <TARGET> <LINK_FLAGS> <OBJECTS>")
> +endif ()

Thin archives seem like a nice idea. How about moving it to OptionsCommon.cmake, though? (and removing it from OptionsGTK)
Comment 6 Ryuan Choi 2014-02-09 15:26:43 PST
Created attachment 223653 [details]
Patch
Comment 7 Ryuan Choi 2014-02-09 15:31:02 PST
(In reply to comment #5)
> (From update of attachment 223650 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=223650&action=review
> 
> > Source/cmake/OptionsEfl.cmake:120
> > +if (${CMAKE_SYSTEM_NAME} STREQUAL "Linux")
> > +    set(CMAKE_CXX_ARCHIVE_CREATE "<CMAKE_AR> cruT <TARGET> <LINK_FLAGS> <OBJECTS>")
> > +    set(CMAKE_C_ARCHIVE_CREATE "<CMAKE_AR> cruT <TARGET> <LINK_FLAGS> <OBJECTS>")
> > +    set(CMAKE_CXX_ARCHIVE_APPEND "<CMAKE_AR> ruT <TARGET> <LINK_FLAGS> <OBJECTS>")
> > +    set(CMAKE_C_ARCHIVE_APPEND "<CMAKE_AR> ruT <TARGET> <LINK_FLAGS> <OBJECTS>")
> > +endif ()
> 
> Thin archives seem like a nice idea. How about moving it to OptionsCommon.cmake, though? (and removing it from OptionsGTK)

Sure, I uploaded like you mentioned.
Comment 8 Philippe Normand 2014-03-05 00:26:34 PST
Ping?

Also adding Martin who's been working a lot on the CMake build recently.
Comment 9 Ryuan Choi 2014-03-05 16:51:56 PST
Comment on attachment 223653 [details]
Patch

thanks
Comment 10 WebKit Commit Bot 2014-03-05 17:22:37 PST
Comment on attachment 223653 [details]
Patch

Clearing flags on attachment: 223653

Committed r165149: <http://trac.webkit.org/changeset/165149>
Comment 11 WebKit Commit Bot 2014-03-05 17:22:42 PST
All reviewed patches have been landed.  Closing bug.