| Summary: | Cmake for static build of libjavascriptcoregtk and libwebkit2gtk fails: install TARGETS given no ARCHIVE DESTINATION for static library target "WebKit". | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | WebKit | Reporter: | Didier Spaier <didier> | ||||||||||||||||||
| Component: | WebKitGTK | Assignee: | Nobody <webkit-unassigned> | ||||||||||||||||||
| Status: | RESOLVED WONTFIX | ||||||||||||||||||||
| Severity: | Normal | CC: | bugs-noreply, didier, mcatanzaro, sfroberg13, stefan.froberg | ||||||||||||||||||
| Priority: | P2 | ||||||||||||||||||||
| Version: | WebKit Local Build | ||||||||||||||||||||
| Hardware: | PC | ||||||||||||||||||||
| OS: | Linux | ||||||||||||||||||||
| Attachments: |
|
||||||||||||||||||||
|
Description
Didier Spaier
2018-03-13 08:07:06 PDT
You're trying to do something unsupported that nobody has ever managed to do successfully before, so hiring a software developer would certainly be advisable. Notice that in bug #181695, the developer did not manage to get JavaScript working at all. WebKit is intended to be installed as a shared library; if you build it as a static library, you're going to effectively have 4-5 different copies of it installed, one static linked into each secondary process, so the already-huge installed size will balloon. You're also disabling a ton of unsupported hidden build flags, so even if you were trying to do a shared library build, I wouldn't expect it to work. (In reply to Didier Spaier from comment #0) > CMake Error at Source/WebKit/CMakeLists.txt:996 (install): > install TARGETS given no ARCHIVE DESTINATION for static library target > "WebKit". I guess it wants you to tell it where to install it.
Add this to Source/WebKit/CMakeLists.txt
install(TARGETS WebKit WebProcess NetworkProcess
ARCHIVE DESTINATION "${LIB_INSTALL_DIR}"
LIBRARY DESTINATION "${LIB_INSTALL_DIR}"
RUNTIME DESTINATION "${LIBEXEC_INSTALL_DIR}"
)
Actually, I managed to get static JavaScriptCore work, but it was only after I switched static musl compiled *Process files with static glibc compiled process files. Then it worked.
But still as today, have no clue why the musl-compiled three *Process files caused the problem .....
(In reply to Michael Catanzaro from comment #1) > You're trying to do something unsupported that nobody has ever managed to do > successfully before I stand corrected! :) Well, at I did manage to build 2.18.6 at least as shared with this:
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
-DCMAKE_INSTALL_PREFIX=/usr \
-DCMAKE_SKIP_RPATH=ON \
-DENABLE_DRAG_SUPPORT=OFF \
-DENABLE_GEOLOCATION=OFF \
-DENABLE_ICONDATABASE=OFF \
-DENABLE_INDEXED_DATABASE=OFF \
-DENABLE_NOTIFICATIONS=OFF \
-DENABLE_PLUGIN_PROCESS_GTK2=OFF \
-DENABLE_SPELLCHECK=OFF \
-DENABLE_VIDEO=OFF \
-DENABLE_WEBDRIVER=OFF \
-DENABLE_WEB_AUDIO=OFF \
-DENABLE_WEB_CRYPTO=OFF \
-DLIBEXEC_INSTALL_DIR=/usr/lib${LIBDIRSUFFIX}/webkit2gtk-4.0 \
-DLIB_INSTALL_DIR=/usr/lib${LIBDIRSUFFIX} \
-DPORT=GTK \
-DUSE_LIBHYPHEN=OFF \
-DUSE_LIBNOTIFY=OFF \
-DUSE_LIBSECRET=OFF \
-DUSE_SYSTEM_MALLOC=ON \
..
I will try Stefan's code snippet and, if I succeed, will check the size.
If it's too big, I will ship it as shared but in in a "hidden" path and use a wrapper script that sets LD_LIBRARY_PATH then exec yelp-binary.
Ugly workaround maybe, but that works
I'll need some days to try, then will report the outcome.
Thanks to both of you.
BTW Michael, few weeks(month?) ago I talked with Konstantin Tokarev about Qt port of webkit and asked if it will support webkit 2 eventually and he said yes. So does that mean that Qt port will also need those three separate *Process files (WebkitStorageProcess, WebkitNetworkProcess and WebkitWebProcess) ? Also, what actually is the WebKitStorageProcess used for ? Sqlite database for cookies? Could it be eliminated (and save space) if one developes private browser that never stores cookies to disk but keeps them in RAM? (In reply to Stefan Fröberg from comment #5) > BTW Michael, few weeks(month?) ago I talked with Konstantin Tokarev about Qt > port of webkit and asked if it will support webkit 2 eventually and he said > yes. > So does that mean that Qt port will also need those three separate *Process > files (WebkitStorageProcess, WebkitNetworkProcess and WebkitWebProcess) ? If applications opt-in to the new API, yes. I recommend switching to using Konstantin's fork of QtWebKit as your upstream, as all major distros have already done. > Also, what actually is the WebKitStorageProcess used for ? Sqlite database > for cookies? > Could it be eliminated (and save space) if one developes private browser > that never stores cookies to disk but keeps them in RAM? Nope, it's mandatory. Storage process is used for databases and for service workers. Not for cookies. You're not going to save a significant amount of space... unless you try static linking WebKit into it. :P Hi Stefan! Indeed it is tricky to get static builds to work (as you have already discovered!), so I wanted to chime in to give you a couple of possible alternatives that you may want to consider: * Build WebKitGTK+ as dynamic libraries, using a custom installation prefix, e.g. with “-DCMAKE_INSTALL_PREFIX=/usr/libexec/yelp-webkit”, disabling the WebKitGTK+ features that Yelp does not need to work, and then linking Yelp to the libraries in the custom prefix. * Use something like Flatpak which is precisely intended for bundling applications, where application bundles can have their dependencies included with them. You could have a custom Flatpak build of Yelp for your distribution which includes a custom build of WebKitGTK+ inside the application bundle. Any of the above will get you out of the “build a static WebKitGTK+” minefield. That being said, we are not against using static builds of WebKitGTK+ but as Michael already mentioned, it is not a particularly attractive option: static builds prevent the multiple processes that are needed by WebKitGTK+ from sharing the code segments of the shared libraries (which saves both disk space and memory). Created attachment 335727 [details]
patch for static build
Created attachment 335728 [details]
Config log most features enabled)
Created attachment 335729 [details]
Build log (most features enabled).
No luck so far, after having completed the patch as advised by Stefan. CMake ran OK but the build failed. I first ran CMake with the same features as in my previous attempt, then once again this time disabling the features for which deps are missing or too old on my system. The end of the build log is the same in both cases. I attach the new patch, named build_main_libs_static_2.20.0.diff, the config log, named CONFIGLOGSTATIC-2.20.0.full, and the build log, named MAKELOGSTATIC-2.20.0. Before I give up, probably then switching to the first alternative proposed by Adrian as I don't ship flatpak yet in Slint, any clue is still appreciated. I just forgot, here are the options that correspond to my last attempt (most features enabled):
LIBDIRSUFFIX="64"
CFLAGS="-Wno-expansion-to-defined"
CXXFLAGS="-Wno-expansion-to-defined"
CXXFLAGS+=" -fno-delete-null-pointer-checks"
CFLAGS+=" -fno-delete-null-pointer-checks"
cmake \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/usr \
-DENABLE_GEOLOCATION=OFF \
-DENABLE_WEB_AUDIO=OFF \
-DENABLE_VIDEO=OFF \
-DLIBEXEC_INSTALL_DIR=/usr/lib${LIBDIRSUFFIX}/webkit2gtk-4.0 \
-DLIB_INSTALL_DIR=/usr/lib${LIBDIRSUFFIX} \
-DPORT=GTK \
-DUSE_SYSTEM_MALLOC=ON \
-DUSE_WOFF2=OFF \
(In reply to Adrian Perez from comment #7) > Hi Stefan! Indeed it is tricky to get static builds to work (as you > have already discovered!), so I wanted to chime in to give you a couple > of possible alternatives that you may want to consider: > > * Build WebKitGTK+ as dynamic libraries, using a custom installation > prefix, e.g. with “-DCMAKE_INSTALL_PREFIX=/usr/libexec/yelp-webkit”, > disabling the WebKitGTK+ features that Yelp does not need to work, > and then linking Yelp to the libraries in the custom prefix. > > * Use something like Flatpak which is precisely intended for bundling > applications, where application bundles can have their dependencies > included with them. You could have a custom Flatpak build of Yelp > for your distribution which includes a custom build of WebKitGTK+ > inside the application bundle. > What would be the pro's and con's of those approach? Does the user need to install extra stuff for example? Does it need specific support from distro? What Im trying to do is build as self-standing app as possible without needing to worry things crazy things like: glibc versioning, having static binary that is not really static (glibc dlopen thing) and so on.... static build musl library would have allowed me to do that but unfortunatley it's not playing nicely with webkit... > Any of the above will get you out of the “build a static WebKitGTK+” > minefield. That being said, we are not against using static builds of > WebKitGTK+ but as Michael already mentioned, it is not a particularly > attractive option: static builds prevent the multiple processes that > are needed by WebKitGTK+ from sharing the code segments of the shared > libraries (which saves both disk space and memory). Okay, is there anyway that the pulled depencies, at least for some of those three *Process files could be lessened? That would greatly help with the size... Now, for example, this is best I can do: -rwxr-xr-x 1 wizard wizard 95M maali 14 17:09 WebKitWebProcess 95 MB, for one of the three *Process files. that added with the other and the app itself the total size is around 285 - 380 MB ... jeez... (as a side note, Qt port of webkit(1?) is only 78 MB as static) I give my musl-static build another run, try to produce patch and then test that with glibc and see how that works. Also, I found out another thing: icu Checking my libicudata.a I found out that it's around 26 MB. That's all just static data sitting there but AFAIK it's not all necessarily needed (it has fully algorithmic conversion for some of the most important encodings) and could be stripped down. Further is the libicudata.a really needed in all three *Process files? (In reply to Didier Spaier from comment #10) > Created attachment 335729 [details] > Build log (most features enabled). That looks similar to bug I posted sometime ago https://bugs.webkit.org/show_bug.cgi?id=183190 root/WebKit/build/DerivedSources/ForwardingHeaders/wtf/TypeCasts.h:82:80: required from 'typename WTF::match_constness<Source, Target>::type& WTF::downcast(Source&) [with Target = WebCore::Document; Source = WebCore::ScriptExecutionContext; typename WTF::match_constness<Source, Target>::type = WebCore::Document]' /root/WebKit/Source/WebCore/bindings/js/JSWebAnimationCustom.cpp:47:49: required from here /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/type_traits:1512:12: error: invalid use of incomplete type 'class WebCore::Document' struct is_base_of Did not get that error with 2.18.6 however with my gcc 7.3.0 Success. Statically compiled webkitgtk 2.18.6 against musl library. Even JavaScript works now (don't know what I did differently this time) https://www.orwell1984.today/testbrowser.tar.xz unpacked size, whopping 344 MB ... To test just do ./test url (for example: ./test https://encrypted.google.com) Below is my configuration, lot's of stuff disabled, even opengl. CFLAGS="-Wno-expansion-to-defined -fno-strict-aliasing -static" CXXFLAGS="-Wno-expansion-to-defined -fno-strict-aliasing -static" LDFLAGS="-static" cmake -DCMAKE_BUILD_TYPE=Release -DSHARED_CORE=OFF -DCMAKE_INSTALL_PREFIX=/robinhood -DENABLE_X11_TARGET=ON -DENABLE_ACCELERATED_2D_CANVAS=OFF -DCMAKE_INSTALL_FULL_LIBEXECDIR=./ -DCMAKE_SKIP_RPATH=ON -DPORT=GTK -DENABLE_WEBGL=OFF -DUSE_LIBNOTIFY=OFF -DENABLE_SPELLCHECK=OFF -DENABLE_INTROSPECTION=OFF -DENABLE_API_TESTS=OFF -DLIB_INSTALL_DIR=/usr/lib -DUSE_LIBHYPHEN=OFF -DENABLE_MINIBROWSER=OFF -DUSE_SYSTEM_MALLOC=OFF -DENABLE_GEOLOCATION=OFF -DENABLE_GTKDOC=OFF -DENABLE_PLUGIN_PROCESS_GTK2=OFF -DENABLE_NETSCAPE_PLUGIN_API=OFF -DENABLE_OPENGL=OFF -DENABLE_SAMPLING_PROFILER=OFF -DENABLE_TOUCH_EVENTS=OFF -DENABLE_VIDEO=OFF -DENABLE_WEBDRIVER=OFF -DENABLE_WEB_AUDIO=OFF -DENABLE_WEBKIT2=ON -DENABLE_WEBASSEMBLY=OFF -DUSE_THIN_ARCHIVES=OFF -DENABLE_TOOLS=OFF -DENABLE_REMOTE_INSPECTOR=ON -DENABLE_JIT=OFF -DENABLE_ASSEMBLER=OFF -Wno-dev .. Quick and dirty patch for static building following with also 2 patches for building with opengl disabled and only X11 backend left. Created attachment 335818 [details]
Patch for building static webkitgtk
Patch for building static webkitgtk 2.18.6
Created attachment 335819 [details]
Patch for building without OpenGL part 1/2
Created attachment 335820 [details]
Patch for building without OpenGL part 2/2
Forgot one thing. When/if the build get's around 99% then the link.txt needs tweaking for each *Process files to build them successfully. For example, for webkitgtk-2.18.6/build/Source/WebKit/CMakeFiles/StorageProcess.dir/link.txt the following monster was needed (I had to remove any *.so references and wrap the static libs between into -Wl,--start-group ... -Wl,--end-group to solve circular dependency madness... /usr/bin/c++ -Wno-noexcept-type -Wno-maybe-uninitialized -Wno-expansion-to-defined -Wwrite-strings -Wundef -Wpointer-arith -Wmissing-format-attribute -Wformat-security -Wcast-align -Wextra -Wall -Wno-expansion-to-defined -fno-strict-aliasing -static -fno-strict-aliasing -fno-exceptions -std=c++14 -fno-rtti -O3 -DNDEBUG -static -fuse-ld=gold -Wl,--disable-new-dtags -rdynamic CMakeFiles/StorageProcess.dir/StorageProcess/EntryPoint/unix/StorageProcessMain.cpp.o -o ../../bin/WebKitStorageProcess -Wl,--start-group ../../lib/libwebkit2gtk-4.0.a ../../lib/libWebCoreGTK.a ../../lib/libWebCorePlatformGTK.a -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 ../../lib/libWebCoreDerivedSources.a ../../lib/libjavascriptcoregtk-4.0.a ../../lib/libPAL.a ../../lib/libwoff2.a ../../lib/libbrotli.a ../../lib/libxdgmime.a /usr/lib/libcairo.a /usr/lib/libfontconfig.a /usr/lib/libfreetype.a /usr/lib/libharfbuzz.a /usr/lib/libharfbuzz-icu.a /usr/lib/libgcrypt.a /usr/lib/libjpeg.a /usr/lib/libpng.a /usr/lib/libwebp.a /usr/lib/libatk-1.0.a /usr/lib/libcairo.a /usr/lib/libfontconfig.a /usr/lib/libfreetype.a /usr/lib/libharfbuzz.a /usr/lib/libharfbuzz-icu.a /usr/lib/libgcrypt.a /usr/lib/libjpeg.a /usr/lib/libpng.a /usr/lib/libwebp.a /usr/lib/libatk-1.0.a /usr/lib/libgmodule-2.0.a -lsecret-1 -lgio-2.0 -lgobject-2.0 -lglib-2.0 /usr/lib/libsoup-2.4.a /usr/lib/libtasn1.a /usr/lib/libxml2.a /usr/lib/libxslt.a /usr/lib/libsqlite3.a /usr/lib/libX11.a /usr/lib/libXcomposite.a /usr/lib/libXdamage.a /usr/lib/libXrender.a /usr/lib/libXt.a ../../lib/libWTFGTK.a ../../lib/libbmalloc.a -ldl /usr/lib/libicudata.a /usr/lib/libicuuc.a /usr/lib/libgio-2.0.a /usr/lib/libgobject-2.0.a /usr/lib/libz.a /usr/lib/libicui18n.a /usr/lib/libglib-2.0.a -lffi -lpcre -lxcb -lgpg-error -lXau -lXdmcp -lgraphite2 -llzma -lexpat -lbz2 -lxcb-shm -lxcb-render -lpixman-1 -lepoxy -lX11-xcb -lpangoft2-1.0 -lX11 -lXi -lXfixes -lXext -latk-bridge-2.0 -ldbus-1 -latspi -Wl,--end-group Created attachment 335843 [details]
Buid log as shared.
(In reply to Stefan Fröberg from comment #15) > (In reply to Didier Spaier from comment #10) > > Created attachment 335729 [details] > > Build log (most features enabled). > > That looks similar to bug I posted sometime ago > > https://bugs.webkit.org/show_bug.cgi?id=183190 > Did not get that error with 2.18.6 however with my gcc 7.3.0 Yes, it looks very similar to what I get her with gcc 5.5.0. To make sure I tried again using the same options with 2.18.6 (success) and 2.20.0 (failure). I just added USE_WOFF2=OFF for 2.20 as I don't have that installed. Also, with 2.20.0 it took only 2 minutes to get to more than 50% completion. Isn't that strange? Variables and options: LIBDIRSUFFIX="64" CFLAGS="-Wno-expansion-to-defined" CXXFLAGS="-Wno-expansion-to-defined" CXXFLAGS+=" -fno-delete-null-pointer-checks" CFLAGS+=" -fno-delete-null-pointer-checks" cmake \ -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \ -DCMAKE_INSTALL_PREFIX=/usr \ -DCMAKE_SKIP_RPATH=ON \ -DENABLE_DRAG_SUPPORT=OFF \ -DENABLE_GEOLOCATION=OFF \ -DENABLE_ICONDATABASE=OFF \ -DENABLE_INDEXED_DATABASE=OFF \ -DENABLE_NOTIFICATIONS=OFF \ -DENABLE_PLUGIN_PROCESS_GTK2=OFF \ -DENABLE_SPELLCHECK=OFF \ -DENABLE_VIDEO=OFF \ -DENABLE_WEBDRIVER=OFF \ -DENABLE_WEB_AUDIO=OFF \ -DENABLE_WEB_CRYPTO=OFF \ -DLIBEXEC_INSTALL_DIR=/usr/lib${LIBDIRSUFFIX}/webkit2gtk-4.0 \ -DLIB_INSTALL_DIR=/usr/lib${LIBDIRSUFFIX} \ -DPORT=GTK \ -DUSE_LIBHYPHEN=OFF \ -DUSE_LIBNOTIFY=OFF \ -DUSE_LIBSECRET=OFF \ -DUSE_SYSTEM_MALLOC=ON \ -DUSE_WOFF2=OFF \ .. 2>&1| tee ../../CONFIGLOGSHARED-2.20.0 make -j 2 2>&1| tee ../../MAKELOGSHARED-2.20.0|| return 1 I attach the MAKELOGSHARED-2.20.0. So, as far as I can tell there is a regression in 2.20.0 vs 2.18.6. (In reply to Didier Spaier from comment #22) > (In reply to Stefan Fröberg from comment #15) > > (In reply to Didier Spaier from comment #10) > > > Created attachment 335729 [details] > > > Build log (most features enabled). > > > > That looks similar to bug I posted sometime ago > > > > https://bugs.webkit.org/show_bug.cgi?id=183190 > > Did not get that error with 2.18.6 however with my gcc 7.3.0 > > Yes, it looks very similar to what I get her with gcc 5.5.0. > > To make sure I tried again using the same options with 2.18.6 (success) and > 2.20.0 (failure). I just added USE_WOFF2=OFF for 2.20 as I don't have that > installed. Also, with 2.20.0 it took only 2 minutes to get to more than 50% > completion. Isn't that strange? > > Variables and options: > > LIBDIRSUFFIX="64" > CFLAGS="-Wno-expansion-to-defined" > CXXFLAGS="-Wno-expansion-to-defined" > CXXFLAGS+=" -fno-delete-null-pointer-checks" > CFLAGS+=" -fno-delete-null-pointer-checks" > cmake \ > -DCMAKE_BUILD_TYPE=Release \ > -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \ > -DCMAKE_INSTALL_PREFIX=/usr \ > -DCMAKE_SKIP_RPATH=ON \ > -DENABLE_DRAG_SUPPORT=OFF \ > -DENABLE_GEOLOCATION=OFF \ > -DENABLE_ICONDATABASE=OFF \ > -DENABLE_INDEXED_DATABASE=OFF \ > -DENABLE_NOTIFICATIONS=OFF \ > -DENABLE_PLUGIN_PROCESS_GTK2=OFF \ > -DENABLE_SPELLCHECK=OFF \ > -DENABLE_VIDEO=OFF \ > -DENABLE_WEBDRIVER=OFF \ > -DENABLE_WEB_AUDIO=OFF \ > -DENABLE_WEB_CRYPTO=OFF \ > -DLIBEXEC_INSTALL_DIR=/usr/lib${LIBDIRSUFFIX}/webkit2gtk-4.0 \ > -DLIB_INSTALL_DIR=/usr/lib${LIBDIRSUFFIX} \ > -DPORT=GTK \ > -DUSE_LIBHYPHEN=OFF \ > -DUSE_LIBNOTIFY=OFF \ > -DUSE_LIBSECRET=OFF \ > -DUSE_SYSTEM_MALLOC=ON \ > -DUSE_WOFF2=OFF \ > .. 2>&1| tee ../../CONFIGLOGSHARED-2.20.0 > make -j 2 2>&1| tee ../../MAKELOGSHARED-2.20.0|| return 1 > > I attach the MAKELOGSHARED-2.20.0. > > So, as far as I can tell there is a regression in 2.20.0 vs 2.18.6. Yes, that's for sure. BTW, I made a custom libicudata.a and linked all the stuff against it and saved around 100 MB from the total :-) So now the total size of my test app + 3 *Process files is 251 MB. But there is more, if Im not wrong the whole webkitgtk is compiled with -O3 That adds size of about 30% So next I try how it behaves with -O2 or even -Os and also the total size. Also, I don't see any breakings with encodings so far (knocks wood) after replacing stock libicudata.a (26 MB) with my custom libicudata.a (1.7 MB) Tested with: ./test http://www.qq.com and looks same as the previous bigger version : -) Hmmm...
Should these be added to end of list(APPEND WebCore_LIBRARIES
at Source/WebCore/PlatformGTK.cmake ?
${X11_Xau_LIB}
${X11_Xdmcp_LIB}
${X11_Xext_LIB}
${X11_Xfixes_LIB}
${X11_Xi_LIB}
using -Os and LTO takes away another 100 MB. Now the total size about 153 MB Have to check if size optimizing the depencies the webkitgtk uses bring any supstantial savings .... |