Bug 152183 – [GTK][Mac] gobject-introspection uses wrong shared-library path

• Component: WebKitGTK
• Status: NEW

[Tom Schoonjans]

Hi,

I am the maintainer of the webkitgtk formula for Homebrew. I noticed today that the gobject-introspection typelib files provided by webkitgtk do not work.

I discovered this while running a simple test with gjs, which imports the Webkit2 typelib and results in the following run-time error:

** (gjs:89955): WARNING **: Failed to load shared library '/tmp/webkitgtk20151211-60021-ktyz48/webkitgtk-2.10.4/lib/libwebkit2gtk-4.0.37.dylib' referenced by the typelib: dlopen(/tmp/webkitgtk20151211-60021-ktyz48/webkitgtk-2.10.4/lib/libwebkit2gtk-4.0.37.dylib, 9): image not found

Clearly what happened here is that the build path was used by g-ir-scanner instead of the installation prefix. I confirmed this by inspecting both gir and typelib files

An extract from WebKit2-4.0.gir:

  <namespace name="WebKit2"
             version="4.0"
             shared-library="/tmp/webkitgtk20151211-60021-ktyz48/webkitgtk-2.10.4/lib/libwebkit2gtk-4.0.37.dylib,/tmp/webkitgtk20151211-60021-ktyz48/webkitgtk-2.10.4/lib/libjavascriptcoregtk-4.0.18.dylib"
             c:identifier-prefixes="WebKit"
             c:symbol-prefixes="webkit">

[Michael Catanzaro]

Uh-oh.

On Linux it is:

  <namespace name="WebKit2"
             version="4.0"
             shared-library="libwebkit2gtk-4.0.so.37,libjavascriptcoregtk-4.0.so.18"
             c:identifier-prefixes="WebKit"
             c:symbol-prefixes="webkit">

[Philip Chimento]

Hmm, I can confirm this on my JHbuild setup but never noticed anything wrong because the libraries remained in the location specified by the build path.

I'd assume this is a gobject-introspection bug unless there is anything funny about the way the cmake file calls g-ir-scanner. WebKit's one of the few clients of g-ir-scanner who uses it without libtool, and those code paths don't get exercised a lot. I've exposed other bugs in g-ir-scanner as well by building WebKit.

[Philip Chimento]

Sorry, should have specified using g-ir-scanner without libtool _on Darwin_. Of course it works without libtool on Linux, per comment 1.

[Jeremy Huddleston Sequoia]

With current trunk, it seems to be working for me:

/opt/local/share/gir-1.0 $ cat WebKit2WebExtension-4.0.gir | grep dylib
             shared-library="/opt/local/lib/libwebkit2gtk-4.0.37.dylib,/opt/local/lib/libjavascriptcoregtk-4.0.18.dylib"

/opt/local/share/gir-1.0 $ cat WebKit2-4.0.gir | grep dylib
             shared-library="/opt/local/lib/libwebkit2gtk-4.0.37.dylib,/opt/local/lib/libjavascriptcoregtk-4.0.18.dylib"

Is this still an issue for you?

[Tom Schoonjans]

It is still present in 2.10.8 for sure.

I cannot really clone the webkit repository as I am currently using a really bad internet connection. Which commit introduces the patch?

Did the fix make it into the 2.11.91 unstable release?

[Andrea Giammarchi]

I can confirm this bug is still present. On top of that, also the javascriptcore library is missing in a similar fashion (looking for /tmp/some-folder as well)

[Jeremy Huddleston Sequoia]

Andrea, what version of WebKit are you using?  What version of gobject-introspection?

I don't see the issue with trunk WebKit and gobject-introspection 1.46.0.  This implies that it is likely either fixed or a bug in gobject-introspection.

[Andrea Giammarchi]

On Linux (ArchLinux on x86) these are used versions and it works OK

$ sudo pacman -Qs webkit2gtk
local/webkit2gtk 2.10.8-1
    GTK+ Web content engine library

$ sudo pacman -Qs gobject-introspection
local/gobject-introspection 1.46.0-1
    Introspection system for GObject-based libraries
local/gobject-introspection-runtime 1.46.0-1
    Introspection system for GObject-based libraries - runtime files

$ sudo pacman -Qs gjs
local/gjs 1.44.0-2
    Javascript Bindings for GNOME

On OSX though, the webkitgtk 2.10.8 (bottle) installed via `brew install webkitgtk` and using gobject-introspection 1.46.0_1 and gjs 1.44.0 I always have the same error:

Failed to load shared library '/tmp/webkit ....' for libwebkit2gtk-4.0.37.dylib

Failed to load shared library '/tmp/webkit.....' for libjavascriptcoregtk-4.0.18.dylib


Not sure how to try via homebrew the trunk version.

[Jeremy Huddleston Sequoia]

Ok, then that seems to suggest that it is either fixed in a newer version on WebKit (I don't know what revision introduced the fix) or it is a HomeBrew-specific issue since we don't see the problem in MacPorts with the same version of gobject-introspection.

[Andrea Giammarchi]

but then we are full circle, since Homebrew chaps said it's a WebKit thing:
https://github.com/Homebrew/homebrew/issues/47000

I've noticed a while ago MacPorts was OK but it had to build it in place, right?

Is there anything you see here that looks suspicious and could be the cause ?
https://github.com/Homebrew/homebrew/blob/master/Library/Formula/webkitgtk.rb

Thanks for any sort of outcome/hint/help

[Jeremy Huddleston Sequoia]

We are not full circle there.  If HomeBrew is correct, and it was a WebKit bug, then it was likely fixed by something in the past months since 2.10 branched.

I'm not sure what you mean by "but it had to build it in place" in regards to MacPorts.  Can you elaborate, and I'd be happy to answer.

Nothing looks terribly off in the Homebrew recipe, but there are certainly a lot of darwin-specific bugs in 2.10.x that have since been fixed in 2.11.x, so I'm a bit surprised by the simplicity (lack of workarounds and patches) of the recipe.

[Jeremy Huddleston Sequoia]

FWIW, the 2.11.92 build coming out of MacPorts doesn’t have this issue, so Tom, you can probably give that tarball a shot since you can’t checkout svn.

[Tom Schoonjans]

I gave the 2.11.92 tarball a try and ended up with this error, which seems related to gobject-introspection:

cd /tmp/webkitgtk20160320-19252-18gsk2z/webkitgtk-2.11.92/Tools/MiniBrowser/gtk && /usr/local/Cellar/cmake/3.5.0/bin/cmake -E cmake_link_script CMakeFiles/MiniBrowser.dir/link.txt --verbose=1
/usr/local/Library/ENV/4.3/clang++    -std=c++11 -Qunused-arguments -DNDEBUG -fno-exceptions -fno-strict-aliasing -fno-rtti -Wl,-search_paths_first -Wl,-headerpad_max_install_names  CMakeFiles/MiniBrowser.dir/__/__/__/DerivedSources/MiniBrowser/BrowserMarshal.c.o CMakeFiles/MiniBrowser.dir/BrowserCellRendererVariant.c.o CMakeFiles/MiniBrowser.dir/BrowserDownloadsBar.c.o CMakeFiles/MiniBrowser.dir/BrowserSearchBar.c.o CMakeFiles/MiniBrowser.dir/BrowserSettingsDialog.c.o CMakeFiles/MiniBrowser.dir/BrowserWindow.c.o CMakeFiles/MiniBrowser.dir/main.c.o  -o ../../../bin/MiniBrowser  ../../../lib/libwebkit2gtk-4.0.37.13.5.dylib -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -lintl /usr/local/lib/libglib-2.0.dylib ../../../lib/libWebCoreGTK.a ../../../lib/libGObjectDOMBindings.a ../../../lib/libWebCorePlatformGTK.a -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -lintl ../../../lib/libjavascriptcoregtk-4.0.18.3.7.dylib ../../../lib/libWTFGTK.a /usr/local/opt/icu4c/lib/libicui18n.dylib ../../../lib/libbmalloc.a -lpthread /usr/local/opt/icu4c/lib/libicuuc.dylib /usr/local/lib/libjpeg.dylib /usr/local/lib/libpng.dylib -lwebp /usr/local/lib/libatk-1.0.dylib /usr/local/lib/libcairo.dylib /usr/local/lib/libenchant.dylib /usr/local/lib/libfontconfig.dylib /usr/local/lib/libfreetype.dylib /usr/local/lib/libgio-2.0.dylib /usr/local/lib/libgmodule-2.0.dylib /usr/local/lib/libgobject-2.0.dylib /usr/local/lib/libglib-2.0.dylib /usr/local/lib/libharfbuzz.dylib /usr/local/lib/libharfbuzz-icu.dylib /usr/local/lib/libsoup-2.4.dylib /usr/lib/libxml2.dylib /usr/lib/libxslt.dylib /usr/local/opt/sqlite/lib/libsqlite3.dylib /usr/lib/libz.dylib -Wl,-all_load 
[100%] Built target WebProcess
[100%] Built target NetworkProcess
[100%] Built target DatabaseProcess
[100%] Built target webkit2gtkinjectedbundle
[100%] Built target MiniBrowser
Command '['/private/tmp/webkitgtk20160320-19252-18gsk2z/webkitgtk-2.11.92/Source/WebKit2/tmp-introspectLI4AJn/WebKit2-4.0', '--introspect-dump=/private/tmp/webkitgtk20160320-19252-18gsk2z/webkitgtk-2.11.92/Source/WebKit2/tmp-introspectLI4AJn/functions.txt,/private/tmp/webkitgtk20160320-19252-18gsk2z/webkitgtk-2.11.92/Source/WebKit2/tmp-introspectLI4AJn/dump.xml']' returned non-zero exit status -5
make[2]: *** [WebKit2-4.0.gir] Error 1
make[1]: *** [Source/WebKit2/CMakeFiles/WebKit2-4-gir.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
Command '['/private/tmp/webkitgtk20160320-19252-18gsk2z/webkitgtk-2.11.92/Source/WebKit2/tmp-introspectYI2JVz/WebKit2-4.0', '--introspect-dump=/private/tmp/webkitgtk20160320-19252-18gsk2z/webkitgtk-2.11.92/Source/WebKit2/tmp-introspectYI2JVz/functions.txt,/private/tmp/webkitgtk20160320-19252-18gsk2z/webkitgtk-2.11.92/Source/WebKit2/tmp-introspectYI2JVz/dump.xml']' returned non-zero exit status -5
make[2]: *** [WebKit2-4.0.gir] Error 1
make[1]: *** [Source/WebKit2/CMakeFiles/WebKit2WebExtension-4-gir.dir/all] Error 2
make: *** [all] Error 2

[Jeremy Huddleston Sequoia]

Tom, you failed to include the actual error.  As the build is parallelized, the actual error is likely many lines above in your log.  It is also not this bug, so you should probably file a separate ticket.

[Jeremy Huddleston Sequoia]

Note that if you're on El Capitan, the issue might be happening because g-ir-scanner has a shebang of '#!/usr/bin/env /path/to/python'.  The use of usr/bin/env with SIP causes DYLD_LIBRRY_PATH to NOT get passed down in the environment.

We addressed that in MacPorts by fixing the shebang in gobject-introspection scripts to be "#!/path/to/macports/prefix/bin/python".  You should do something similar, or it can also be worked around in the webkit-gtk build system (but of course other projects will still fail) doing:

-        ${INTROSPECTION_SCANNER}
+        ${PYTHON_EXECUTABLE} ${INTROSPECTION_SCANNER}

in the appropriate places in Source/WebKit2/PlatformGTK.cmake

[Michael Catanzaro]

(In reply to comment #15)
> Note that if you're on El Capitan, the issue might be happening because
> g-ir-scanner has a shebang of '#!/usr/bin/env /path/to/python'.  The use of
> usr/bin/env with SIP causes DYLD_LIBRRY_PATH to NOT get passed down in the
> environment.

Hm, #!/usr/bin/env python shebangs are not permitted in Debian, so I think it's a bug that g-ir-scanner does this. The shebang should be either #!/usr/bin/python2 or #!/usr/bin/python3 depending on which python it wants, or #!/usr/bin/python if it's compatible with both pythons.

[Jeremy Huddleston Sequoia]

If you use /usr/bin/python, you will have issues because DYLD will disallow the DYLD_LIBRARY_PATH being passed to the system executable.  You will need to use your own python.

[Tom Schoonjans]

I was already using Homebrew's python but I set the shebang in g-ir-scanner explicitly to this python anyway: same error came through :-(

These are actually the only lines in the log that showed something went wrong. Nothing else popped up...

[Andrea Giammarchi]

FYI, and FWIW, after trying `brew install --build-from-source webkitgtk` and about 524 minutes on El Capitan and a Mac Mini, the problem is **still the same** even if no error was apparently shown during building the whole package.

Moreover, even if I fix the link through a temporary folder, there is a Fontconfig error that makes it crash.

[Jeremy Huddleston Sequoia]

Tom, please file a new bug report for your issue and attach the full build log.  Please reference it from here, so I notice it.

Andrea, I don't think Homebrew updated to 2.11.92, so I don't think that is unexpected.

[Andrea Giammarchi]

Jeremy you're right, but that's the unstable branch, I hoped 2.10.9 stable brought in fixes.

I guess I have to wait for 2.12 stable before trying to build again in case of problems.

Best Regards

[Andrea Giammarchi]

So, 2.12 stable got released. I will try to test it manually but please confirm me that would include patches to fix this issue. Thanks.

[Jeremy Huddleston Sequoia]

Andrea, I'm not aware of any patches (or what change may have fixed this).  The only data points that I have are that master and 2.11.x (and thus 2.12.0) don't have this issue on MacPorts.