Bug 170500 - [GTK][META] Bumping all modules of jhbuild to the latest stable version
Summary: [GTK][META] Bumping all modules of jhbuild to the latest stable version
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebKit Gtk (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on: 170540 170942 171595 171918 173538 173592 173710
Blocks:
  Show dependency treegraph
 
Reported: 2017-04-05 02:19 PDT by Fujii Hironori
Modified: 2017-06-22 06:07 PDT (History)
7 users (show)

See Also:


Attachments
WIP (76.91 KB, patch)
2017-04-06 04:15 PDT, Carlos Garcia Campos
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Fujii Hironori 2017-04-05 02:19:41 PDT
[GTK] Bumping all modules of jhbuild to the latest stable version 

Here are the KaL's new proposal:

  [webkit-gtk] Policy for versions of deps used in internal jhbuild for testing
  https://lists.webkit.org/pipermail/webkit-gtk/2016-October/002819.html

Let's do this.
Comment 1 Carlos Garcia Campos 2017-04-05 03:34:36 PDT
Ok, this is my initial proposal:

pixman: 0.32.6 -> 0.34.0
cairo: 1.14.2 -> 1.14.8
freetype: 2.4.11 -> 2.6.3
harfbuzz: 1.3.3 -> 1.4.2
libffi: 3.1 (do we really need this in the internal jhbuild? glib depends on 3.0 I think we can remove this one)
gdk-pixbuf: 2.30 -> 2.36.6
librsvg: 2.36.1 -> 2.40.16
gtk+: 3.16.4 -> 3.22.11
glib: 2.44.1 -> 2.52.0
glib-networking: 2.41.4 -> 2.50.0
libsoup: 2.57.1 (this is the latest already)
fontconfig: 2.11.1 (this is recent enough)
gnome-icon-theme: 3.2.1 (do we really need this)
gnome-icon-theme-symbolic: 3.2.1 (do we really need this)
gnome-themes-standard: 3.6.0 (do we really need this)
atk: 2.15.4 -> 2.24.0
at-spi2-core: 2.15.4 -> 2.24.0
at-spi2-atk: 2.15.4 -> 2.24.0
libxml2: 2.9.1 (do we really need this)
libxslt: 1.1.29 (do we really need this)
orc: 0.4.17
gstreamer: 1.8.0 -> 1.10.4
gst-plugins-base: 1.8.0 -> 1.10.4
gst-plugins-good: 1.8.0 -> 1.10.4
gst-plugins-bad: 1.8.0 -> 1.10.4
gst-libav: 1.8.0 -> 1.10.4
xserver: 1.16.4 -> 1.19.2
wayland: 1.8.1 -> 1.12.0
weston: 1.8.0 -> 1.12.0
gtk-doc: 1.25 (this is recent enough)
libdrm: 2.4.65 -> 2.4.74
mesa: 11.0.6 -> 13.0.6
llvm: 3.7.0 -> 3.8.0
gsettings-desktop-schemas: 3.16 (do we really need this)
shared-mime-info: 1.5 -> 1.8
icu: 55.1 -> 57.1
pango: 1.36.8 -> 1.40.4
libinput: 1.2.4 -> 1.6.3

I wanted to bump all at once to check the tests, but maybe we can do it in two steps and leave gst related modules to a second patch.
Comment 2 Adrian Perez 2017-04-05 07:12:38 PDT
The plan looks good to me. I have added a few comments below.

(In reply to Carlos Garcia Campos from comment #1)
> Ok, this is my initial proposal:
> 
> [...]
> libffi: 3.1 (do we really need this in the internal jhbuild? glib depends on
> 3.0 I think we can remove this one)

This can go away. Even Debian Stable has version 3.1 of libffi
nowadays: https://packages.debian.org/jessie/libffi-dev

> [...]
> librsvg: 2.36.1 -> 2.40.16

Just in case someone else wonders why not using 2.41.0: that version is
the first one including some parts rewritten in Rust. Staying in 2.40.x
saves us from building a Rust toolchain and from requiring it installed
system-wide. IMHO for now 2.40.16 is a good tradeoff between updating
and not requiring one extra complex dependency.

> [...]
> gnome-icon-theme: 3.2.1 (do we really need this)
> gnome-icon-theme-symbolic: 3.2.1 (do we really need this)
> gnome-themes-standard: 3.6.0 (do we really need this)

Using GTK+ from JHBuild does not fall-back well to using the versions
of these packages installed system-wide. I think that's the reason why
we have been keeping those in the moduleset.

> [...]
> libxml2: 2.9.1 (do we really need this)

Debian stable has this version already (exactly 2.9.1, actually). I would
remove it from the JHBuild moduleset.

> libxslt: 1.1.29 (do we really need this)

Our CMake files check for version 1.1.7, and Debian stable has 1.1.28.
Most likely we can remove it from the JHBuild.

> [...]
> orc: 0.4.17

Is there any reason to not update this? Could it be removed, even?
(Debian stable has 0.4.22, I guess we have to keep it in the JHBuild
if changing it could affect the output from tests, but that seems
unlikely. Dunno, I am not sure about this one.)
Comment 3 Carlos Garcia Campos 2017-04-05 09:40:12 PDT
I'll split it in more parts, there are too many failures to track at once, I'm getting ~5600 failures :-P
Comment 4 Carlos Garcia Campos 2017-04-05 09:44:56 PDT
(In reply to Carlos Garcia Campos from comment #3)
> I'll split it in more parts, there are too many failures to track at once,
> I'm getting ~5600 failures :-P

Maybe most of them are only because of freetype, though. I'll check it.
Comment 5 Carlos Garcia Campos 2017-04-05 10:10:00 PDT
Expected to fail, but passed: (127)
Regressions: Unexpected text-only failures (5443)
Regressions: Unexpected image-only failures (24)
Comment 6 Carlos Alberto Lopez Perez 2017-04-05 13:03:10 PDT
Feel free to upload the WIP patch here. I can help to test it.
Comment 7 Carlos Garcia Campos 2017-04-06 04:15:41 PDT
Created attachment 306375 [details]
WIP

This is the complete patch
Comment 8 Carlos Alberto Lopez Perez 2017-04-20 16:17:40 PDT
I get this(In reply to Carlos Garcia Campos from comment #7)
> Created attachment 306375 [details]
> WIP
> 
> This is the complete patch

I also get around ~5k new failures:

Regressions: Unexpected text-only failures (5507)

A quick look at the results shows that there are many tests that just need a rebaseline (just some pixel differences on the RenderBlocks). But many others are clear real failures.

And having to manually go through 5k results to check which ones need a rebaseline or not is crazy.

I'm thinking in creating some utility that can automatically tell is the test just need a rebaseline (by checking that the diff only changes pixel numbers on the render blocks) or if it needs manual inspection because it may be a potential failure.

I wonder if we already have something like that??? How the Mac port handles rebaselines when they release a new version of OSX?? Any idea?
Comment 9 Michael Catanzaro 2017-04-20 21:10:15 PDT
What we need to do is upgrade the libraries one-by-one so we can figure out if a particular library update broke tests and get the updates that don't break tests out of the way. I bet only three or four of those libraries actually break any tests. Trying to take them all on all at the same time is clearly not going to work. Maybe it would work in the future if we upgrade more regularly, but doing several years' worth of upgrades for all dependencies all at the same time is just not going to work.
Comment 10 Carlos Garcia Campos 2017-04-20 23:43:21 PDT
(In reply to Michael Catanzaro from comment #9)
> What we need to do is upgrade the libraries one-by-one so we can figure out
> if a particular library update broke tests and get the updates that don't
> break tests out of the way.

Yes, that's the plan, and the reason why this is a meta bug and the patch just a reference to be split.

> I bet only three or four of those libraries
> actually break any tests.

cairo and freetype are my bets. Other deps like libxml, gtk+, etc. will introduce failures for sure, not not that many.

> Trying to take them all on all at the same time is
> clearly not going to work.

We already realized, see comment 3.

> Maybe it would work in the future if we upgrade
> more regularly, but doing several years' worth of upgrades for all
> dependencies all at the same time is just not going to work.

Indeed, we should upgrade at least once per cycle, at the very beginning.
Comment 11 Carlos Alberto Lopez Perez 2017-04-21 03:19:09 PDT
(In reply to Michael Catanzaro from comment #9)
> What we need to do is upgrade the libraries one-by-one so we can figure out
> if a particular library update broke tests and get the updates that don't
> break tests out of the way. I bet only three or four of those libraries
> actually break any tests. Trying to take them all on all at the same time is
> clearly not going to work. Maybe it would work in the future if we upgrade
> more regularly, but doing several years' worth of upgrades for all
> dependencies all at the same time is just not going to work.

Broking them one by one is not going to cut the 5000 layout failures down. Only split them.

So I still see the need for some tool that can help to automatically difference the diffs that are just minor pixel differences on the render block fom something else.

This will prove useful as time passes because we intend to keep upgrading the moduleset much more frequently
Comment 12 Fujii Hironori 2017-04-21 03:51:19 PDT
What about this? https://trac.webkit.org/wiki/RebaselineServer
Comment 13 Carlos Garcia Campos 2017-06-22 06:07:01 PDT
I've upgraded cairo and pixman and there aren't differences in test results. So, it was freetype alone the one causing more than 5000 failures. libxml2 is still pending too, there were regressions with 1.9.4 that has been recently fixed in git, so we could either switch to use git or wait for a new libxml2 release.