Bug 93264

Summary: [jhbuild] Add gnutls 3.0.22
Product: WebKit Reporter: Kangil Han <kangil.han>
Component: WebKit EFLAssignee: Kangil Han <kangil.han>
Status: RESOLVED WONTFIX    
Severity: Normal CC: cdumez, gns, gyuyoung.kim, lucas.de.marchi, mpakulavelrutka, mrobinson, pnormand, rakuco, tmpsantos
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Other   
OS: Linux   
Attachments:
Description Flags
patch
none
patch
none
patch
none
patch gns: review-, rakuco: commit-queue-

Description Kangil Han 2012-08-06 06:22:09 PDT
I was stuck on below error while jhbuild.

"""
configure: error: libsoup requires glib-networking for TLS support.

If you are building a package, you can pass --disable-tls-check to
allow building libsoup anyway (since glib-networking is not actually
required at compile time), but you should be sure to add a runtime
dependency on it.
*** Error during phase configure of libsoup: ########## Error running ./configure --prefix /home/hankangil/dev/WebKit/webkit-efl/WebKit/WebKitBuild/Dependencies/Root --libdir '/home/hankangil/dev/WebKit/webkit-efl/WebKit/WebKitBuild/Dependencies/Root/lib' --without-gnome --disable-introspection --disable-static --disable-gtk-doc  *** [10/22]
"""
Comment 1 Kangil Han 2012-08-06 06:30:00 PDT
Created attachment 156681 [details]
patch
Comment 2 Thiago Marcos P. Santos 2012-08-06 06:55:09 PDT
The problem here is that many developers are using Ubuntu 10.10 and the default gnutls it is not enough.

gnutls was removed from jhbuild at bug 90643. I would be nice to to hear from Raphael first.

IMO is fine either way.
Comment 3 Kangil Han 2012-08-06 07:09:22 PDT
(In reply to comment #2)
> The problem here is that many developers are using Ubuntu 10.10 and the default gnutls it is not enough.
> 
> gnutls was removed from jhbuild at bug 90643. I would be nice to to hear from Raphael first.
> 
> IMO is fine either way.

Yes, I totally agree with you.
I also want to get feedback from rakuco since he would have good knowledge on this matter.

FYI, I am using Ubuntu 11.10(32 bit) and have installed 'libgnutls-dev'.
Comment 4 Gyuyoung Kim 2012-08-06 07:19:08 PDT
Personally, I don't like to add new module to jhbuild because of build time increase. However, if we need to support older Ubuntu, though we recommend to use Ubuntu 12.04, I agree to land this patch. But, if you wanna land this patch, we also need to consider GTK port as well.
Comment 5 Kangil Han 2012-08-06 22:09:16 PDT
(In reply to comment #4)
> Personally, I don't like to add new module to jhbuild because of build time increase. However, if we need to support older Ubuntu, though we recommend to use Ubuntu 12.04, I agree to land this patch. But, if you wanna land this patch, we also need to consider GTK port as well.

Thanks!
I've found gtk port is also using libsoup, different version though.
I will run gtk jhbuild and get back with result.
Comment 6 Kangil Han 2012-08-07 02:36:25 PDT
I had build error of 'libsoup' on gtk port.
=> configure: error: libsoup requires glib-networking for TLS support.

I did one more test:
  if I insert 'addpath('PKG_CONFIG_PATH', os.path.join(os.sep, 'usr', _libdir, 'i386-linux-gnu', 'pkgconfig'))' line on 'jhbuildrc' to include pkgconfig for apt-get installed 'guntls-dev', then 'glib-networking' complains version of 'gnutls-dev' is lower than '2.10.10'. On ubuntu 11.10, 'gnutls-dev' version is '2.10.5'.
Comment 7 Raphael Kubo da Costa (:rakuco) 2012-08-07 06:43:39 PDT
I oppose adding gnutls back to jhbuild.modules because version 2 will not build with recent versions of glibc (which is why I created bug 90643 in the first place), and version 3 was a royal PITA to get to build on all platforms in that bug report.

When I discussed this issue with kov at the time bug 90643 was being reviewed, we came to the conclusion that we also require other bleeding edge dependencies which are not in jhbuild as well. Plus, WebKit trunk itself is a fast-moving beast which is not even guaranteed to run with some seemingly old platforms (which would be the case of Ubuntu 10.10, for example).

I agree that Ubuntu 11.10 should be recent enough for things to still work, OTOH in this specific case gnutls is causing problems which make it bad to have in our official jhbuild.modules in trunk.

One possible solution is for people to build their own gnutls with a custom jhbuild.modules: both Tools/efl/jhbuildrc and Tools/gtk/jhbuildrc respect the WEBKIT_EXTRA_MODULESETS and WEBKIT_EXTRA_MODULES environment variables [1] which you can then use to specify extra modules or modulesets (aka jhbuild.modules-like files) you want to build.

[1] http://trac.webkit.org/changeset/105519
Comment 8 Kangil Han 2012-08-07 17:01:40 PDT
(In reply to comment #7)
> I oppose adding gnutls back to jhbuild.modules because version 2 will not build with recent versions of glibc (which is why I created bug 90643 in the first place), and version 3 was a royal PITA to get to build on all platforms in that bug report.
> 
> When I discussed this issue with kov at the time bug 90643 was being reviewed, we came to the conclusion that we also require other bleeding edge dependencies which are not in jhbuild as well. Plus, WebKit trunk itself is a fast-moving beast which is not even guaranteed to run with some seemingly old platforms (which would be the case of Ubuntu 10.10, for example).
> 
> I agree that Ubuntu 11.10 should be recent enough for things to still work, OTOH in this specific case gnutls is causing problems which make it bad to have in our official jhbuild.modules in trunk.
> 
> One possible solution is for people to build their own gnutls with a custom jhbuild.modules: both Tools/efl/jhbuildrc and Tools/gtk/jhbuildrc respect the WEBKIT_EXTRA_MODULESETS and WEBKIT_EXTRA_MODULES environment variables [1] which you can then use to specify extra modules or modulesets (aka jhbuild.modules-like files) you want to build.
> 
> [1] http://trac.webkit.org/changeset/105519

I personally feel subtle that Ubuntu version is constrained by 'glibc'. However, using bleeding edge libraries is okay for me. So, if we use version 3, i.e. 'gnutls' 3.0.21, what kind of challenges would we have? Why don't we include other most recently released libraries to jhbuild?
Comment 9 Raphael Kubo da Costa (:rakuco) 2012-08-07 22:32:31 PDT
(In reply to comment #8)
> I personally feel subtle that Ubuntu version is constrained by 'glibc'.
Sorry, I couldn't understand what you meant here.

> However, using bleeding edge libraries is okay for me. So, if we use version 3, i.e. 'gnutls' 3.0.21, what kind of challenges would we have?
Hmm, related to what? If version 3 is used there should be no additional trouble.

> Why don't we include other most recently released libraries to jhbuild?
Even though there is no formal guideline for this, we've been trying to keep the list of software managed by jhbuild down to a minimum that enables us to get the same set of results for layout tests. Increasing the list of software in jhbuild means doing your distro's packaging system's job, and different distros/operating systems have might build their software in different ways, with different patches and build options, so it's better not to try to do that as well.
Comment 10 Kangil Han 2012-08-07 22:58:18 PDT
(In reply to comment #9)
> > However, using bleeding edge libraries is okay for me. So, if we use version 3, i.e. 'gnutls' 3.0.21, what kind of challenges would we have?
> Hmm, related to what? If version 3 is used there should be no additional trouble.
> 

Did you mean it is okay if I put gnutls version 3 to jhbuild?
Comment 11 Raphael Kubo da Costa (:rakuco) 2012-08-07 23:03:18 PDT
(In reply to comment #10)
> (In reply to comment #9)
> > > However, using bleeding edge libraries is okay for me. So, if we use version 3, i.e. 'gnutls' 3.0.21, what kind of challenges would we have?
> > Hmm, related to what? If version 3 is used there should be no additional trouble.
> 
> Did you mean it is okay if I put gnutls version 3 to jhbuild?

It is OK if you use it locally in your own module set. If it is added to the general jhbuild.modules we maintain in trunk, it is also OK, but we had quite a lot of trouble getting it to build on the GTK bots, so we just decided to remove it from jhbuild.
Comment 12 Kangil Han 2012-08-07 23:31:04 PDT
(In reply to comment #11)
> It is OK if you use it locally in your own module set. If it is added to the general jhbuild.modules we maintain in trunk, it is also OK, but we had quite a lot of trouble getting it to build on the GTK bots, so we just decided to remove it from jhbuild.

I know you had trouble in bug 90643.
However, some of my colleagues are also using Ubuntu 11.10, so I want to find best solution for more developers.
I would really appreciate it if you would share general gtk bot information with me.
Thanks! :-)
Comment 13 Raphael Kubo da Costa (:rakuco) 2012-08-07 23:44:29 PDT
(In reply to comment #12)
> I would really appreciate it if you would share general gtk bot information with me.
> Thanks! :-)

It is all in the GTK EWS results in bug 90643 -- gnutls 3 requires nettle and libgmp-dev; first the EWS maintainers had to install libgmp-dev on the bots, but later nettle 2.5 mysteriously failed to build on Philippe's bot.

You can grab one of the earlier patches I sent for that bug and try using the nettle and gnutls modules there in a separate module set.
Comment 14 Kangil Han 2012-08-08 00:29:26 PDT
(In reply to comment #13)
> 
> You can grab one of the earlier patches I sent for that bug and try using the nettle and gnutls modules there in a separate module set.

Thanks!
I will refer to it. :-)
Comment 15 Kangil Han 2012-08-08 20:12:00 PDT
Created attachment 157376 [details]
patch

Let's see if bot would be happy with gnutls version 3.
Wish me a luck!
Comment 16 Kangil Han 2012-08-09 00:03:03 PDT
Build bots are happy with gnutls 3 version. LoL
Formal/Informal review please!
Comment 17 Raphael Kubo da Costa (:rakuco) 2012-08-09 15:08:34 PDT
It's good that it worked, however this looks like the patches I sent in bug 90643 (without some configuration flag updates). Isn't it OK to just use it locally instead of reverting what we did in the other bug?
Comment 18 Kangil Han 2012-08-09 18:14:21 PDT
(In reply to comment #17)
> It's good that it worked, however this looks like the patches I sent in bug 90643 (without some configuration flag updates). Isn't it OK to just use it locally instead of reverting what we did in the other bug?

Sorry, I don't understand fully what you meant.
One thing clear is that this patch would mitigate efforts to set extra module for some developers.
Comment 19 Kangil Han 2012-08-11 02:22:53 PDT
Dear rakuco,

I am not 100% sure what I've understood correctly with respect to you last comment regarding 'reverting' on this patch. However, I don't think this patch would revert your BUG 90643 because the major change of this patch would be updating libsoup and gnutls with the most recent ones. Plus, this patch would help my colleagues, who use Ubuntu 11.10 and couldn't upgrade to 12.04 right now, build WebKit EFL without any configuration. I will update things that I might have missed on the patch if you would continue informal review.

Thanks in advance.
Comment 20 Raphael Kubo da Costa (:rakuco) 2012-08-12 17:40:11 PDT
(In reply to comment #19)
> I am not 100% sure what I've understood correctly with respect to you last comment regarding 'reverting' on this patch. However, I don't think this patch would revert your BUG 90643 because the major change of this patch would be updating libsoup and gnutls with the most recent ones.

Well, it does revert r124094 in the sense that it reintroduces gnutls to both EFL and GTK's jhbuild.modules.

I'll address specific points in the patch itself.
Comment 21 Raphael Kubo da Costa (:rakuco) 2012-08-12 17:44:17 PDT
Comment on attachment 157376 [details]
patch

View in context: https://bugs.webkit.org/attachment.cgi?id=157376&action=review

> Tools/ChangeLog:8
> +        Adding gnutls version 3 will fix build error on glibc 2.16.0 and lower version of Ubuntu 12.04

This is very vague, and it also looks like you are referring to the same error while you actually have two separate issues. Please mention you are reverting r124094 and explain why and which errors are being avoided.

> Tools/efl/jhbuild.modules:-14
> -      <dep package="glib"/>
> -      <dep package="glib-networking"/>

Why? It doesn't seem related to the purpose of this patch.

> Tools/efl/jhbuild.modules:139
> -    <branch module="/pub/gnome/sources/libsoup/2.39/libsoup-2.39.4.1.tar.xz" version="2.39.4.1"
> -            repo="ftp.gnome.org"
> -            hash="sha256:0740c3f6e393d31f87f166f0c6d11de5ebaa013d382c59ed924a1384c575612e"
> -            md5sum="895ed1e0c949a2ce50aeaeb58661bfc3"/>
> +    <branch module="/pub/gnome/sources/libsoup/2.39/libsoup-2.39.5.tar.xz" version="2.39.5"
> +            repo="ftp.gnome.org"/>

Ditto.

> Tools/efl/jhbuild.modules:143
> +             autogenargs="--enable-ld-version-script --enable-cxx --without-lzo --with-libgcrypt">

Please watch the configuration output of gnutls: the gcrypt backend is not enabled in version 3, and LZO support has been dropped, so --without-lzo and --with-libgcrypt have no effect.

> Tools/efl/jhbuild.modules:145
> +      <dep package="libgcrypt"/>

Not needed.

> Tools/gtk/jhbuild.modules:-16
> -      <dep package="glib"/>
> -      <dep package="glib-networking"/>

Same comments I made to the EFL file.

> Tools/gtk/jhbuild.modules:197
> +  <autotools id="libgpg-error" autogen-sh="configure">
> +    <branch module="/gcrypt/libgpg-error/libgpg-error-1.10.tar.bz2" version="1.10"
> +             repo="gnupg.org"
> +             hash="sha256:520629b4568b5c29b1991c8ffc267c8bdee5f223c7333c42a651b56f9b1c5431"
> +             md5sum="736a03daa9dc5873047d4eb4a9c22a16"/>
> +  </autotools>
> +
> +  <autotools id="libgcrypt" autogen-sh="./autogen.sh; configure">
> +    <dependencies>
> +      <dep package="libgpg-error"/>
> +    </dependencies>
> +    <branch module="/gcrypt/libgcrypt/libgcrypt-1.5.0.tar.bz2" version="1.5.0"
> +             repo="gnupg.org"
> +             hash="sha256:4b62fc516004940a0571025401a0581d49199f1a76dfb5ce6fd63f50db8173fa"
> +             md5sum="693f9c64d50c908bc4d6e01da3ff76d8"/>
> +  </autotools>

Not needed.

> Tools/gtk/jhbuild.modules:200
> +  <autotools id="gnutls"
> +             autogenargs="--enable-ld-version-script --enable-cxx --without-lzo --with-libgcrypt">

Same comments I made to the EFL file.
Comment 22 Raphael Kubo da Costa (:rakuco) 2012-08-12 17:45:04 PDT
Before proceeding, I'd really like to hear kov's and the other GTK guys' opinions, as r124094 was discussed with them and this changes GTK's jhbuild.modules as well.
Comment 23 Kangil Han 2012-08-13 08:34:52 PDT
Created attachment 158007 [details]
patch
Comment 24 Kangil Han 2012-08-13 08:37:18 PDT
kov: Would you please let us know about your opinion on this patch?

rakuco: Thanks for the review and done!
Comment 25 Chris Dumez 2012-08-13 08:44:15 PDT
Comment on attachment 158007 [details]
patch

You forgot to update "FIND_PACKAGE(LibSoup2 2.39.4.1 REQUIRED)" line in Source/cmake/OptionsEfl.cmake.
Comment 26 Raphael Kubo da Costa (:rakuco) 2012-08-13 09:13:46 PDT
Comment on attachment 158007 [details]
patch

I still don't understand why libsoup's version was bumped -- we only do that when we require a bug fix or some new API present in a more recent version.
Comment 27 Chris Dumez 2012-08-13 09:39:02 PDT
(In reply to comment #26)
> (From update of attachment 158007 [details])
> I still don't understand why libsoup's version was bumped -- we only do that when we require a bug fix or some new API present in a more recent version.

Good point :) I was assuming that Kangil had a good reason. If there is a good reason, it should be documented in the Changelog.
Comment 28 Kangil Han 2012-08-13 17:46:49 PDT
(In reply to comment #27)
> (In reply to comment #26)
> > (From update of attachment 158007 [details] [details])
> > I still don't understand why libsoup's version was bumped -- we only do that when we require a bug fix or some new API present in a more recent version.
> 
> Good point :) I was assuming that Kangil had a good reason. If there is a good reason, it should be documented in the Changelog.

rakuco: Okay, I will keep libsoup version.

chris: Nope, I don't have a nice reason. It was a general approach. Latest version would be better. -_-
Comment 29 Kangil Han 2012-08-13 17:58:18 PDT
Created attachment 158174 [details]
patch
Comment 30 Kangil Han 2012-08-15 19:14:42 PDT
kov: Would you please let us know your opinion on this patch?

rakuco, chris: Any more comments?
Comment 31 Chris Dumez 2012-08-15 21:54:40 PDT
Comment on attachment 158174 [details]
patch

For the EFL side, LGTM. You'll need kov or mrobinson to review the GTK side.
Comment 32 Kangil Han 2012-08-15 23:03:59 PDT
(In reply to comment #31)
> (From update of attachment 158174 [details])
> For the EFL side, LGTM. You'll need kov or mrobinson to review the GTK side.

Thanks!
I will try to keep in touch with GTK bot maintainers. :-)
Comment 33 Kangil Han 2012-08-16 01:30:26 PDT
mrobinson, kov: Would you please review this patch?
Comment 34 Michal Pakula vel Rutka 2012-08-16 06:55:47 PDT
I have applied your patch on Ubuntu 11.10 and during configuring gnutls I got this:

checking whether to use nettle... yes
checking for libnettle... no
configure: error: 
  ***
  *** Libnettle 2.4 was not found. You must compile nettle with gmp support.

while nettle was built and installed without any problems.

Of course after installing nettle-dev via apt-get everything was built without any problems.
Comment 35 Raphael Kubo da Costa (:rakuco) 2012-08-16 08:15:58 PDT
(In reply to comment #34)
> Of course after installing nettle-dev via apt-get everything was built without any problems.

Which means the nettle from jhbuild is most likely not being used.

I'd be interested in the configuration output for both nettle and gnutls; does it work if you remove your nettle package and keep libgmp-dev around?
Comment 36 Kangil Han 2012-08-16 17:23:15 PDT
(In reply to comment #35)
> (In reply to comment #34)
> > Of course after installing nettle-dev via apt-get everything was built without any problems.
> 
> Which means the nettle from jhbuild is most likely not being used.
> 
> I'd be interested in the configuration output for both nettle and gnutls; does it work if you remove your nettle package and keep libgmp-dev around?

rakuco: Thanks for kind explanation!

michal: I've experienced my local 'LD_LIBRARY_PATH', had used before jhbuild has adopted, disturbed package installation. If rakuco's guide doesn't work for you, then please disable your local settings('LD_LIBRARY_PATH', 'LD_INCLUDE_PATH' and 'PKG_CONFIG_PATH') and do try again? :-)
Comment 37 Michal Pakula vel Rutka 2012-08-17 00:46:18 PDT
(In reply to comment #35)
> (In reply to comment #34)
> > Of course after installing nettle-dev via apt-get everything was built without any problems.
> 
> Which means the nettle from jhbuild is most likely not being used.
> 
> I'd be interested in the configuration output for both nettle and gnutls; does it work if you remove your nettle package and keep libgmp-dev around?

I have removed nettle-dev and and dependent packages and then installing only libgmp-dev worked for me.
Comment 38 Gustavo Noronha (kov) 2012-08-23 08:34:36 PDT
Comment on attachment 158174 [details]
patch

My opinion on this is I would prefer not to reintroduce gnutls, p11kit and other things. We do not usually do things based on Ubuntu versions - many of us do not use ubuntu, but Debian or Fedora. That said, if we do worry about Ubuntu it's the LTS versions that matter, because it's what companies usually have a policy of running. So I think it would be a good idea to try and keep WebKitGTK+ buildable in Ubuntu 12.04 from now on, but I would prefer not to try and support 11.10.

What I suggest for your colleagues that cannot upgrade is to use the jhbuild modules override facility to add these libraries to their build systems. See here: http://trac.webkit.org/wiki/BuildingGtk#Extendingthejhbuildenvironment

r- for the gtk changes, I obviously do not oppose changing EFL if desired =)
Comment 39 Raphael Kubo da Costa (:rakuco) 2012-08-23 09:23:47 PDT
I concur with kov's opinion here. I'd rather get the list of things built by jhbuild down to the bare minimum, as one packaging system already gives one enough headaches :-)

One other thing mrobison suggested a few days ago was for interested parties to create a PPA for this kind of dependency, which can help if people don't really like tinkering with jhbuild.
Comment 40 Raphael Kubo da Costa (:rakuco) 2012-09-07 06:48:24 PDT
Can we close this one now?
Comment 41 Kangil Han 2012-09-07 06:54:11 PDT
Sure rakuco.
I will close this bug.