Bug 108330 – [CMake] Use thin archives if building on Linux for non-shared-core debug builds

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Jussi Kukkonen (jku)]

I've been trying to package webkit-efl lately, and noticed that we could use "thin archives" to get around the problem of .a archives growing over 4GB by using "thin archives". When thin archives are used while building static webcore_efl, the size of the archive goes from >4GB to <100MB.

"""
gnu ar can optionally create a thin archive, which contains a symbol index and references to the original copies of the member files of the archives. Such an archive is useful for building libraries for use within a local build, where the relocatable objects are expected to remain available, and copying the contents of each object would only waste time and space. Thin archives are also flattened, so that adding one or more archives to a thin archive will add the elements of the nested archive individually. The paths to the elements of the archive are stored relative to the archive itself.
"""

It's totally possible to leave this up to the distribution builders, but I think we could make it default as well (when building without SHARED_CORE) since I don't see why someone would not want this -- webkit-gtk already does the same, see bug 107400.

In practice the change would mean redefining these: 
  CMAKE_CXX_ARCHIVE_CREATE="<CMAKE_AR> cruT <TARGET> <LINK_FLAGS> <OBJECTS>"
  CMAKE_CXX_ARCHIVE_APPEND="<CMAKE_AR> ruT  <TARGET> <LINK_FLAGS> <OBJECTS>"
The only additions to default commands are "uT", but it seems we have to redefine the whole thing for that.

I'm still testing this, but filing a bug to keep track of it. CCing Kubo as he promised to take a look and comment whether it looks sane to him.

[Jussi Kukkonen (jku)]

Just saw thiago (CCd) on #webkit-gtk:

tmpsantos:   CXXLD  libwebkitgtk-3.0.la
tmpsantos: libtool: link: warning: `-version-info/-version-number' is ignored for convenience libraries
tmpsantos: ar: `x' cannot be used on thin archives.

So there may still be some problems with this, at least in some cases.

[Tobias Mueller]

Running into this, too:

make  all-am
make[1]: Entering directory `/home/muelli/svn/gnome2/WebKit'
/usr/bin/mkdir -p ./.deps/DerivedSources
/bin/sh ./libtool  --tag=CXX   --mode=link g++ -fvisibility-inlines-hidden -fno-rtti -g3 -pipe -O2 -Wno-c++11-compat -Wno-c++11-extensions -O2 -version-info 18:1:18 -Wl,--version-script,./Source/autotools/symbols.filter  -L/opt/gnome2/lib64 -Wl,-O0 -Wl,--enable-new-dtags -Wl,--sort-common -Wl,--as-needed -o libwebkitgtk-3.0.la   Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-AcceleratedCompositingContextClutter.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-AcceleratedCompositingContextGL.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-AcceleratedCompositingContextCairo.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-AssertMatchingEnums.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-ChromeClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-ContextMenuClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-DeviceMotionClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-DeviceOrientationClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-DocumentLoaderGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-DragClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-DumpRenderTreeSupportGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-EditorClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-FrameLoaderClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-FrameNetworkingContextGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-GeolocationClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-GtkAdjustmentWatcher.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-InspectorClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-NavigatorContentUtilsClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-PlatformStrategiesGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-UserMediaClientGtk.lo Source/WebKit/gtk/WebCoreSupport/libwebkitgtk_3_0_la-WebViewInputMethodFilter.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitapplicationcache.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitdownload.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkiterror.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitfavicondatabase.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitfilechooserrequest.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitgeolocationpolicydecision.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitglobals.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkithittestresult.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkiticondatabase.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitnetworkrequest.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitnetworkresponse.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitsecurityorigin.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitsoupauthdialog.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitspellchecker.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitspellcheckerenchant.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitversion.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitviewportattributes.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebbackforwardlist.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebdatabase.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebdatasource.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebframe.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebhistoryitem.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebinspector.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebnavigationaction.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebpolicydecision.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebresource.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebplugin.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebplugindatabase.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebsettings.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebview.lo Source/WebKit/gtk/webkit/libwebkitgtk_3_0_la-webkitwebwindowfeatures.lo DerivedSources/webkit/libwebkitgtk_3_0_la-webkitenumtypes.lo DerivedSources/webkit/libwebkitgtk_3_0_la-webkitmarshal.lo  -lpthread libWebCore.la libWebCorePlatform.la libWebCoreModules.la libWebCoreDOM.la libWebCoreGtk.la   libjavascriptcoregtk-3.0.la -lcairo       -L/opt/gnome2/lib64 -lcairo -lfontconfig -lfreetype -lharfbuzz   -L/opt/gnome2/lib64 -lgailutil-3 -lgtk-3 -lgdk-3 -latk-1.0 -lgio-2.0 -lpangocairo-1.0 -lgdk_pixbuf-2.0 -lcairo-gobject -lpango-1.0 -lcairo -lgobject-2.0 -lglib-2.0     -Wl,--export-dynamic -pthread -L/opt/gnome2/lib64 -lgmodule-2.0 -lgthread-2.0 -lrt -lgio-2.0 -lgobject-2.0 -lglib-2.0    -L/opt/gnome2/lib64 -lgtk-3 -lgdk-3 -latk-1.0 -lgio-2.0 -lpangocairo-1.0 -lgdk_pixbuf-2.0 -lcairo-gobject -lpango-1.0 -lcairo -lgobject-2.0 -lglib-2.0   -ljpeg -L/opt/gnome2/lib64 -lsecret-1 -lgio-2.0 -lgobject-2.0 -lglib-2.0   -L/opt/gnome2/lib64 -lsoup-2.4 -lgio-2.0 -lgobject-2.0 -lglib-2.0   -L/opt/gnome2/lib64 -lxml2    -lGL -ldl -L/opt/gnome2/lib64 -lpangoft2-1.0 -lpango-1.0 -lfreetype -lfontconfig -lgobject-2.0 -lglib-2.0   -lpng15   -lsqlite3   -licui18n -licuuc -licudata  -lwebp -lXcomposite   -lXdamage -lXfixes   -lXrender -lX11   -lXt -lX11      -lz   -lrt 
libtool: link: warning: `-version-info/-version-number' is ignored for convenience libraries
libtool: link: rm -fr  .libs/libwebkitgtk-3.0.lax
libtool: link: (cd .libs/libwebkitgtk-3.0.lax/libWebCore.a && ar x "/home/muelli/svn/gnome2/WebKit/./.libs/libWebCore.a")
ar: `x' cannot be used on thin archives.
make[1]: *** [libwebkitgtk-3.0.la] Error 1
make[1]: Target `all-am' not remade because of errors.
make[1]: Leaving directory `/home/muelli/svn/gnome2/WebKit'
make: *** [all] Error 2


[jhbuild] muelli@bigbox ~/svn/gnome2/WebKit [±:master?] $ head config.log 
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.

It was created by WebKitGTK configure 1.11.5, which was
generated by GNU Autoconf 2.68.  Invocation command line was

  $ ./configure --prefix /opt/gnome2 --libdir /opt/gnome2/lib64 --enable-introspection --disable-silent-rules --with-gstreamer=1.0 --disable-static --disable-scrollkeeper --disable-gtk-doc --disable-docs --enable-gtk-doc=no --enable-gtk-doc-html=no --enable-gtk-doc-pdf=no --disable-webkit1 --disable-spellcheck --disable-glx --disable-egl --disable-gles2 --disable-gamepad --disable-video --disable-mediastream --disable-xslt --disable-geolocation --disable-svg --disable-svg-fonts --disable-web-audio --disable-coverage --disable-webgl --disable-gtk-doc-html --disable-accelerated-compositing LDFLAGS=-L/opt/gnome2/lib64 -Wl,-O0 -Wl,--enable-new-dtags -Wl,--sort-common -Wl,--as-needed CXXFLAGS=-g3 -pipe -O2 PKG_CONFIG_PATH=/opt/gnome2/lib64/pkgconfig:/opt/gnome2/share/pkgconfig:/usr/share/pkgconfig:/usr/lib64/pkgconfig:/usr/lib/pkgconfig:/opt/gnome/lib/pkgconfig --no-create --no-recursion

[Ryuan Choi]

Will you apply it ?

I think that it can be good idea.

[Ryuan Choi]

Created attachment 223650 [details]
Patch

[Sergio Correia (qrwteyrutiyoup)]

Comment on attachment 223650 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=223650&action=review

> Source/cmake/OptionsEfl.cmake:120
> +if (${CMAKE_SYSTEM_NAME} STREQUAL "Linux")
> +    set(CMAKE_CXX_ARCHIVE_CREATE "<CMAKE_AR> cruT <TARGET> <LINK_FLAGS> <OBJECTS>")
> +    set(CMAKE_C_ARCHIVE_CREATE "<CMAKE_AR> cruT <TARGET> <LINK_FLAGS> <OBJECTS>")
> +    set(CMAKE_CXX_ARCHIVE_APPEND "<CMAKE_AR> ruT <TARGET> <LINK_FLAGS> <OBJECTS>")
> +    set(CMAKE_C_ARCHIVE_APPEND "<CMAKE_AR> ruT <TARGET> <LINK_FLAGS> <OBJECTS>")
> +endif ()

Thin archives seem like a nice idea. How about moving it to OptionsCommon.cmake, though? (and removing it from OptionsGTK)

[Ryuan Choi]

Created attachment 223653 [details]
Patch

[Ryuan Choi]

(In reply to comment #5)
> (From update of attachment 223650 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=223650&action=review
> 
> > Source/cmake/OptionsEfl.cmake:120
> > +if (${CMAKE_SYSTEM_NAME} STREQUAL "Linux")
> > +    set(CMAKE_CXX_ARCHIVE_CREATE "<CMAKE_AR> cruT <TARGET> <LINK_FLAGS> <OBJECTS>")
> > +    set(CMAKE_C_ARCHIVE_CREATE "<CMAKE_AR> cruT <TARGET> <LINK_FLAGS> <OBJECTS>")
> > +    set(CMAKE_CXX_ARCHIVE_APPEND "<CMAKE_AR> ruT <TARGET> <LINK_FLAGS> <OBJECTS>")
> > +    set(CMAKE_C_ARCHIVE_APPEND "<CMAKE_AR> ruT <TARGET> <LINK_FLAGS> <OBJECTS>")
> > +endif ()
> 
> Thin archives seem like a nice idea. How about moving it to OptionsCommon.cmake, though? (and removing it from OptionsGTK)

Sure, I uploaded like you mentioned.

[Philippe Normand]

Ping?

Also adding Martin who's been working a lot on the CMake build recently.

[Ryuan Choi]

Comment on attachment 223653 [details]
Patch

thanks

[WebKit Commit Bot]

Comment on attachment 223653 [details]
Patch

Clearing flags on attachment: 223653

Committed r165149: <http://trac.webkit.org/changeset/165149>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.