Bug 93264 – [jhbuild] Add gnutls 3.0.22

• Component: WebKit EFL
• Status: RESOLVED WONTFIX

[Kangil Han]

I was stuck on below error while jhbuild.

"""
configure: error: libsoup requires glib-networking for TLS support.

If you are building a package, you can pass --disable-tls-check to
allow building libsoup anyway (since glib-networking is not actually
required at compile time), but you should be sure to add a runtime
dependency on it.
*** Error during phase configure of libsoup: ########## Error running ./configure --prefix /home/hankangil/dev/WebKit/webkit-efl/WebKit/WebKitBuild/Dependencies/Root --libdir '/home/hankangil/dev/WebKit/webkit-efl/WebKit/WebKitBuild/Dependencies/Root/lib' --without-gnome --disable-introspection --disable-static --disable-gtk-doc  *** [10/22]
"""

[Kangil Han]

Created attachment 156681 [details]
patch

[Thiago Marcos P. Santos]

The problem here is that many developers are using Ubuntu 10.10 and the default gnutls it is not enough.

gnutls was removed from jhbuild at bug 90643. I would be nice to to hear from Raphael first.

IMO is fine either way.

[Kangil Han]

(In reply to comment #2)
> The problem here is that many developers are using Ubuntu 10.10 and the default gnutls it is not enough.
> 
> gnutls was removed from jhbuild at bug 90643. I would be nice to to hear from Raphael first.
> 
> IMO is fine either way.

Yes, I totally agree with you.
I also want to get feedback from rakuco since he would have good knowledge on this matter.

FYI, I am using Ubuntu 11.10(32 bit) and have installed 'libgnutls-dev'.

[Gyuyoung Kim]

Personally, I don't like to add new module to jhbuild because of build time increase. However, if we need to support older Ubuntu, though we recommend to use Ubuntu 12.04, I agree to land this patch. But, if you wanna land this patch, we also need to consider GTK port as well.

[Kangil Han]

(In reply to comment #4)
> Personally, I don't like to add new module to jhbuild because of build time increase. However, if we need to support older Ubuntu, though we recommend to use Ubuntu 12.04, I agree to land this patch. But, if you wanna land this patch, we also need to consider GTK port as well.

Thanks!
I've found gtk port is also using libsoup, different version though.
I will run gtk jhbuild and get back with result.

[Kangil Han]

I had build error of 'libsoup' on gtk port.
=> configure: error: libsoup requires glib-networking for TLS support.

I did one more test:
  if I insert 'addpath('PKG_CONFIG_PATH', os.path.join(os.sep, 'usr', _libdir, 'i386-linux-gnu', 'pkgconfig'))' line on 'jhbuildrc' to include pkgconfig for apt-get installed 'guntls-dev', then 'glib-networking' complains version of 'gnutls-dev' is lower than '2.10.10'. On ubuntu 11.10, 'gnutls-dev' version is '2.10.5'.

[Raphael Kubo da Costa (:rakuco)]

I oppose adding gnutls back to jhbuild.modules because version 2 will not build with recent versions of glibc (which is why I created bug 90643 in the first place), and version 3 was a royal PITA to get to build on all platforms in that bug report.

When I discussed this issue with kov at the time bug 90643 was being reviewed, we came to the conclusion that we also require other bleeding edge dependencies which are not in jhbuild as well. Plus, WebKit trunk itself is a fast-moving beast which is not even guaranteed to run with some seemingly old platforms (which would be the case of Ubuntu 10.10, for example).

I agree that Ubuntu 11.10 should be recent enough for things to still work, OTOH in this specific case gnutls is causing problems which make it bad to have in our official jhbuild.modules in trunk.

One possible solution is for people to build their own gnutls with a custom jhbuild.modules: both Tools/efl/jhbuildrc and Tools/gtk/jhbuildrc respect the WEBKIT_EXTRA_MODULESETS and WEBKIT_EXTRA_MODULES environment variables [1] which you can then use to specify extra modules or modulesets (aka jhbuild.modules-like files) you want to build.

[1] http://trac.webkit.org/changeset/105519

[Kangil Han]

(In reply to comment #7)
> I oppose adding gnutls back to jhbuild.modules because version 2 will not build with recent versions of glibc (which is why I created bug 90643 in the first place), and version 3 was a royal PITA to get to build on all platforms in that bug report.
> 
> When I discussed this issue with kov at the time bug 90643 was being reviewed, we came to the conclusion that we also require other bleeding edge dependencies which are not in jhbuild as well. Plus, WebKit trunk itself is a fast-moving beast which is not even guaranteed to run with some seemingly old platforms (which would be the case of Ubuntu 10.10, for example).
> 
> I agree that Ubuntu 11.10 should be recent enough for things to still work, OTOH in this specific case gnutls is causing problems which make it bad to have in our official jhbuild.modules in trunk.
> 
> One possible solution is for people to build their own gnutls with a custom jhbuild.modules: both Tools/efl/jhbuildrc and Tools/gtk/jhbuildrc respect the WEBKIT_EXTRA_MODULESETS and WEBKIT_EXTRA_MODULES environment variables [1] which you can then use to specify extra modules or modulesets (aka jhbuild.modules-like files) you want to build.
> 
> [1] http://trac.webkit.org/changeset/105519

I personally feel subtle that Ubuntu version is constrained by 'glibc'. However, using bleeding edge libraries is okay for me. So, if we use version 3, i.e. 'gnutls' 3.0.21, what kind of challenges would we have? Why don't we include other most recently released libraries to jhbuild?

[Raphael Kubo da Costa (:rakuco)]

(In reply to comment #8)
> I personally feel subtle that Ubuntu version is constrained by 'glibc'.
Sorry, I couldn't understand what you meant here.

> However, using bleeding edge libraries is okay for me. So, if we use version 3, i.e. 'gnutls' 3.0.21, what kind of challenges would we have?
Hmm, related to what? If version 3 is used there should be no additional trouble.

> Why don't we include other most recently released libraries to jhbuild?
Even though there is no formal guideline for this, we've been trying to keep the list of software managed by jhbuild down to a minimum that enables us to get the same set of results for layout tests. Increasing the list of software in jhbuild means doing your distro's packaging system's job, and different distros/operating systems have might build their software in different ways, with different patches and build options, so it's better not to try to do that as well.

[Kangil Han]

(In reply to comment #9)
> > However, using bleeding edge libraries is okay for me. So, if we use version 3, i.e. 'gnutls' 3.0.21, what kind of challenges would we have?
> Hmm, related to what? If version 3 is used there should be no additional trouble.
> 

Did you mean it is okay if I put gnutls version 3 to jhbuild?

[Raphael Kubo da Costa (:rakuco)]

(In reply to comment #10)
> (In reply to comment #9)
> > > However, using bleeding edge libraries is okay for me. So, if we use version 3, i.e. 'gnutls' 3.0.21, what kind of challenges would we have?
> > Hmm, related to what? If version 3 is used there should be no additional trouble.
> 
> Did you mean it is okay if I put gnutls version 3 to jhbuild?

It is OK if you use it locally in your own module set. If it is added to the general jhbuild.modules we maintain in trunk, it is also OK, but we had quite a lot of trouble getting it to build on the GTK bots, so we just decided to remove it from jhbuild.

[Kangil Han]

(In reply to comment #11)
> It is OK if you use it locally in your own module set. If it is added to the general jhbuild.modules we maintain in trunk, it is also OK, but we had quite a lot of trouble getting it to build on the GTK bots, so we just decided to remove it from jhbuild.

I know you had trouble in bug 90643.
However, some of my colleagues are also using Ubuntu 11.10, so I want to find best solution for more developers.
I would really appreciate it if you would share general gtk bot information with me.
Thanks! :-)

[Raphael Kubo da Costa (:rakuco)]

(In reply to comment #12)
> I would really appreciate it if you would share general gtk bot information with me.
> Thanks! :-)

It is all in the GTK EWS results in bug 90643 -- gnutls 3 requires nettle and libgmp-dev; first the EWS maintainers had to install libgmp-dev on the bots, but later nettle 2.5 mysteriously failed to build on Philippe's bot.

You can grab one of the earlier patches I sent for that bug and try using the nettle and gnutls modules there in a separate module set.

[Kangil Han]

(In reply to comment #13)
> 
> You can grab one of the earlier patches I sent for that bug and try using the nettle and gnutls modules there in a separate module set.

Thanks!
I will refer to it. :-)

[Kangil Han]

Created attachment 157376 [details]
patch

Let's see if bot would be happy with gnutls version 3.
Wish me a luck!

[Kangil Han]

Build bots are happy with gnutls 3 version. LoL
Formal/Informal review please!

[Raphael Kubo da Costa (:rakuco)]

It's good that it worked, however this looks like the patches I sent in bug 90643 (without some configuration flag updates). Isn't it OK to just use it locally instead of reverting what we did in the other bug?

[Kangil Han]

(In reply to comment #17)
> It's good that it worked, however this looks like the patches I sent in bug 90643 (without some configuration flag updates). Isn't it OK to just use it locally instead of reverting what we did in the other bug?

Sorry, I don't understand fully what you meant.
One thing clear is that this patch would mitigate efforts to set extra module for some developers.

[Kangil Han]

Dear rakuco,

I am not 100% sure what I've understood correctly with respect to you last comment regarding 'reverting' on this patch. However, I don't think this patch would revert your BUG 90643 because the major change of this patch would be updating libsoup and gnutls with the most recent ones. Plus, this patch would help my colleagues, who use Ubuntu 11.10 and couldn't upgrade to 12.04 right now, build WebKit EFL without any configuration. I will update things that I might have missed on the patch if you would continue informal review.

Thanks in advance.

[Raphael Kubo da Costa (:rakuco)]

(In reply to comment #19)
> I am not 100% sure what I've understood correctly with respect to you last comment regarding 'reverting' on this patch. However, I don't think this patch would revert your BUG 90643 because the major change of this patch would be updating libsoup and gnutls with the most recent ones.

Well, it does revert r124094 in the sense that it reintroduces gnutls to both EFL and GTK's jhbuild.modules.

I'll address specific points in the patch itself.

[Raphael Kubo da Costa (:rakuco)]

Comment on attachment 157376 [details]
patch

View in context: https://bugs.webkit.org/attachment.cgi?id=157376&action=review

> Tools/ChangeLog:8
> +        Adding gnutls version 3 will fix build error on glibc 2.16.0 and lower version of Ubuntu 12.04

This is very vague, and it also looks like you are referring to the same error while you actually have two separate issues. Please mention you are reverting r124094 and explain why and which errors are being avoided.

> Tools/efl/jhbuild.modules:-14
> -      <dep package="glib"/>
> -      <dep package="glib-networking"/>

Why? It doesn't seem related to the purpose of this patch.

> Tools/efl/jhbuild.modules:139
> -    <branch module="/pub/gnome/sources/libsoup/2.39/libsoup-2.39.4.1.tar.xz" version="2.39.4.1"
> -            repo="ftp.gnome.org"
> -            hash="sha256:0740c3f6e393d31f87f166f0c6d11de5ebaa013d382c59ed924a1384c575612e"
> -            md5sum="895ed1e0c949a2ce50aeaeb58661bfc3"/>
> +    <branch module="/pub/gnome/sources/libsoup/2.39/libsoup-2.39.5.tar.xz" version="2.39.5"
> +            repo="ftp.gnome.org"/>

Ditto.

> Tools/efl/jhbuild.modules:143
> +             autogenargs="--enable-ld-version-script --enable-cxx --without-lzo --with-libgcrypt">

Please watch the configuration output of gnutls: the gcrypt backend is not enabled in version 3, and LZO support has been dropped, so --without-lzo and --with-libgcrypt have no effect.

> Tools/efl/jhbuild.modules:145
> +      <dep package="libgcrypt"/>

Not needed.

> Tools/gtk/jhbuild.modules:-16
> -      <dep package="glib"/>
> -      <dep package="glib-networking"/>

Same comments I made to the EFL file.

> Tools/gtk/jhbuild.modules:197
> +  <autotools id="libgpg-error" autogen-sh="configure">
> +    <branch module="/gcrypt/libgpg-error/libgpg-error-1.10.tar.bz2" version="1.10"
> +             repo="gnupg.org"
> +             hash="sha256:520629b4568b5c29b1991c8ffc267c8bdee5f223c7333c42a651b56f9b1c5431"
> +             md5sum="736a03daa9dc5873047d4eb4a9c22a16"/>
> +  </autotools>
> +
> +  <autotools id="libgcrypt" autogen-sh="./autogen.sh; configure">
> +    <dependencies>
> +      <dep package="libgpg-error"/>
> +    </dependencies>
> +    <branch module="/gcrypt/libgcrypt/libgcrypt-1.5.0.tar.bz2" version="1.5.0"
> +             repo="gnupg.org"
> +             hash="sha256:4b62fc516004940a0571025401a0581d49199f1a76dfb5ce6fd63f50db8173fa"
> +             md5sum="693f9c64d50c908bc4d6e01da3ff76d8"/>
> +  </autotools>

Not needed.

> Tools/gtk/jhbuild.modules:200
> +  <autotools id="gnutls"
> +             autogenargs="--enable-ld-version-script --enable-cxx --without-lzo --with-libgcrypt">

Same comments I made to the EFL file.

[Raphael Kubo da Costa (:rakuco)]

Before proceeding, I'd really like to hear kov's and the other GTK guys' opinions, as r124094 was discussed with them and this changes GTK's jhbuild.modules as well.

[Kangil Han]

Created attachment 158007 [details]
patch

[Kangil Han]

kov: Would you please let us know about your opinion on this patch?

rakuco: Thanks for the review and done!

[Chris Dumez]

Comment on attachment 158007 [details]
patch

You forgot to update "FIND_PACKAGE(LibSoup2 2.39.4.1 REQUIRED)" line in Source/cmake/OptionsEfl.cmake.

[Raphael Kubo da Costa (:rakuco)]

Comment on attachment 158007 [details]
patch

I still don't understand why libsoup's version was bumped -- we only do that when we require a bug fix or some new API present in a more recent version.

[Chris Dumez]

(In reply to comment #26)
> (From update of attachment 158007 [details])
> I still don't understand why libsoup's version was bumped -- we only do that when we require a bug fix or some new API present in a more recent version.

Good point :) I was assuming that Kangil had a good reason. If there is a good reason, it should be documented in the Changelog.

[Kangil Han]

(In reply to comment #27)
> (In reply to comment #26)
> > (From update of attachment 158007 [details] [details])
> > I still don't understand why libsoup's version was bumped -- we only do that when we require a bug fix or some new API present in a more recent version.
> 
> Good point :) I was assuming that Kangil had a good reason. If there is a good reason, it should be documented in the Changelog.

rakuco: Okay, I will keep libsoup version.

chris: Nope, I don't have a nice reason. It was a general approach. Latest version would be better. -_-

[Kangil Han]

Created attachment 158174 [details]
patch

[Kangil Han]

kov: Would you please let us know your opinion on this patch?

rakuco, chris: Any more comments?

[Chris Dumez]

Comment on attachment 158174 [details]
patch

For the EFL side, LGTM. You'll need kov or mrobinson to review the GTK side.

[Kangil Han]

(In reply to comment #31)
> (From update of attachment 158174 [details])
> For the EFL side, LGTM. You'll need kov or mrobinson to review the GTK side.

Thanks!
I will try to keep in touch with GTK bot maintainers. :-)

[Kangil Han]

mrobinson, kov: Would you please review this patch?

[Michal Pakula vel Rutka]

I have applied your patch on Ubuntu 11.10 and during configuring gnutls I got this:

checking whether to use nettle... yes
checking for libnettle... no
configure: error: 
  ***
  *** Libnettle 2.4 was not found. You must compile nettle with gmp support.

while nettle was built and installed without any problems.

Of course after installing nettle-dev via apt-get everything was built without any problems.

[Raphael Kubo da Costa (:rakuco)]

(In reply to comment #34)
> Of course after installing nettle-dev via apt-get everything was built without any problems.

Which means the nettle from jhbuild is most likely not being used.

I'd be interested in the configuration output for both nettle and gnutls; does it work if you remove your nettle package and keep libgmp-dev around?

[Kangil Han]

(In reply to comment #35)
> (In reply to comment #34)
> > Of course after installing nettle-dev via apt-get everything was built without any problems.
> 
> Which means the nettle from jhbuild is most likely not being used.
> 
> I'd be interested in the configuration output for both nettle and gnutls; does it work if you remove your nettle package and keep libgmp-dev around?

rakuco: Thanks for kind explanation!

michal: I've experienced my local 'LD_LIBRARY_PATH', had used before jhbuild has adopted, disturbed package installation. If rakuco's guide doesn't work for you, then please disable your local settings('LD_LIBRARY_PATH', 'LD_INCLUDE_PATH' and 'PKG_CONFIG_PATH') and do try again? :-)

[Michal Pakula vel Rutka]

(In reply to comment #35)
> (In reply to comment #34)
> > Of course after installing nettle-dev via apt-get everything was built without any problems.
> 
> Which means the nettle from jhbuild is most likely not being used.
> 
> I'd be interested in the configuration output for both nettle and gnutls; does it work if you remove your nettle package and keep libgmp-dev around?

I have removed nettle-dev and and dependent packages and then installing only libgmp-dev worked for me.

[Gustavo Noronha (kov)]

Comment on attachment 158174 [details]
patch

My opinion on this is I would prefer not to reintroduce gnutls, p11kit and other things. We do not usually do things based on Ubuntu versions - many of us do not use ubuntu, but Debian or Fedora. That said, if we do worry about Ubuntu it's the LTS versions that matter, because it's what companies usually have a policy of running. So I think it would be a good idea to try and keep WebKitGTK+ buildable in Ubuntu 12.04 from now on, but I would prefer not to try and support 11.10.

What I suggest for your colleagues that cannot upgrade is to use the jhbuild modules override facility to add these libraries to their build systems. See here: http://trac.webkit.org/wiki/BuildingGtk#Extendingthejhbuildenvironment

r- for the gtk changes, I obviously do not oppose changing EFL if desired =)

[Raphael Kubo da Costa (:rakuco)]

I concur with kov's opinion here. I'd rather get the list of things built by jhbuild down to the bare minimum, as one packaging system already gives one enough headaches :-)

One other thing mrobison suggested a few days ago was for interested parties to create a PPA for this kind of dependency, which can help if people don't really like tinkering with jhbuild.

[Raphael Kubo da Costa (:rakuco)]

Can we close this one now?

[Kangil Han]

Sure rakuco.
I will close this bug.