| Summary: | [GTK] Bump required versions of some dependencies | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | WebKit | Reporter: | Carlos Garcia Campos <cgarcia> | ||||||||||||||||
| Component: | WebKitGTK | Assignee: | Nobody <webkit-unassigned> | ||||||||||||||||
| Status: | RESOLVED FIXED | ||||||||||||||||||
| Severity: | Normal | CC: | gustavo, mrobinson, philn, rego+ews, rego, webkit.review.bot, xan.lopez, zan | ||||||||||||||||
| Priority: | P2 | ||||||||||||||||||
| Version: | 528+ (Nightly build) | ||||||||||||||||||
| Hardware: | Unspecified | ||||||||||||||||||
| OS: | Unspecified | ||||||||||||||||||
| Bug Depends on: | 113767 | ||||||||||||||||||
| Bug Blocks: | 104779, 111967 | ||||||||||||||||||
| Attachments: |
|
||||||||||||||||||
|
Description
Carlos Garcia Campos
2013-03-26 01:49:30 PDT
Created attachment 195030 [details]
Patch
Comment on attachment 195030 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=195030&action=review > ChangeLog:23 > + depending on harfbuff. It's Harfbuzz :> > ChangeLog:25 > + - Fontconfig 2.5: required by pango 1.32.0 when using harfbuff. Ditto. It seems it doesn't apply on trunk, I'll submit a new patch for trunk and with the Harfbuzz thing fixed. Created attachment 195035 [details]
Rebased patch
Forgot the Harfbuzz again :-/ Created attachment 195057 [details]
Updated patch.
Added atk 2.5.3 to the jhbuild as required by gtk
Created attachment 195069 [details]
Updated patch
Fixed deps order in jhbuild
Created attachment 195078 [details]
Updated patch
Bumped atk and at-spi2 requirements too as required by gtk 3.6
Comment on attachment 195078 [details] Updated patch View in context: https://bugs.webkit.org/attachment.cgi?id=195078&action=review Looks good. r- only because the new modules in the moduleset. > ChangeLog:10 > + Bump libsoup version to 2.42, since it fixes important security > + issues. libsoup requires a newer version of glib, this patch also > + updates other dependencies: I'm fairly certain that the security issue does not actually depend on the version of libsoup. When landing this do you mind fixing the ChangeLog to avoid future confusion? > Tools/gtk/jhbuild.modules:24 > + <dep package="atk"/> The JHbuild moduleset is only for modules that need a specific version for the layout tests to pass. On the other hand, if a dependency is simply hard to get it should go into the optional moduleset. For instance, Cairo is there because even if you have a recent version of Cairo on your system, your version of Cairo might have rasterization differences from the version the bots are running. I've been very careful lately about adding new ones, since it adds a lot of overhead to compilation. > Tools/gtk/jhbuild.modules:177 > + hash="sha256:9e0e7eb5d3f7401ccf521fbc289fc1fa0923b7c7833729e2ed7696f7b848893e" > + md5sum="a632a38d2983c2a88679672d00940128"/> Lately we've only been listing the SHA and not both the SHA and the md5sum (which is a bit overkill). (In reply to comment #9) > (From update of attachment 195078 [details]) > View in context: https://bugs.webkit.org/attachment.cgi?id=195078&action=review > > Looks good. r- only because the new modules in the moduleset. I we bump the requirements and don't update the jhbuild moduleset, the build will fail for people using the internal jhbuild. > > ChangeLog:10 > > + Bump libsoup version to 2.42, since it fixes important security > > + issues. libsoup requires a newer version of glib, this patch also > > + updates other dependencies: > > I'm fairly certain that the security issue does not actually depend on the version of libsoup. When landing this do you mind fixing the ChangeLog to avoid future confusion? Yes, the issue is in libsoup, although we have a workaround in wk to prevent it for older versions. Anyway, I won't mention it in the changelog to avoid confusions. > > Tools/gtk/jhbuild.modules:24 > > + <dep package="atk"/> > > The JHbuild moduleset is only for modules that need a specific version for the layout tests to pass. On the other hand, if a dependency is simply hard to get it should go into the optional moduleset. For instance, Cairo is there because even if you have a recent version of Cairo on your system, your version of Cairo might have rasterization differences from the version the bots are running. I've been very careful lately about adding new ones, since it adds a lot of overhead to compilation. So, how are people using internal jhbuild going to build wk if configure requires higher versions than the one provided by the jhbuild? > > Tools/gtk/jhbuild.modules:177 > > + hash="sha256:9e0e7eb5d3f7401ccf521fbc289fc1fa0923b7c7833729e2ed7696f7b848893e" > > + md5sum="a632a38d2983c2a88679672d00940128"/> > > Lately we've only been listing the SHA and not both the SHA and the md5sum (which is a bit overkill). Ok, I'll remove them Comment on attachment 195078 [details] Updated patch View in context: https://bugs.webkit.org/attachment.cgi?id=195078&action=review >>> Tools/gtk/jhbuild.modules:24 >>> + <dep package="atk"/> >> >> The JHbuild moduleset is only for modules that need a specific version for the layout tests to pass. On the other hand, if a dependency is simply hard to get it should go into the optional moduleset. For instance, Cairo is there because even if you have a recent version of Cairo on your system, your version of Cairo might have rasterization differences from the version the bots are running. I've been very careful lately about adding new ones, since it adds a lot of overhead to compilation. > > So, how are people using internal jhbuild going to build wk if configure requires higher versions than the one provided by the jhbuild? I'm not sure I understand. I'm not suggesting to not bump the dependencies of the existing modules. I'm suggesting that you do not add the new modules, atk and at-spi2-atk. (In reply to comment #11) > I'm not sure I understand. I'm not suggesting to not bump the dependencies of the existing modules. I'm suggesting that you do not add the new modules, atk and at-spi2-atk. These new modules should likely go into jhbuild-optional.modules. ATK should be updated and at-spi2-atk could be added. This will allow people running distributions where these packages are difficult to maintain and easier way to install them. (In reply to comment #12) > (In reply to comment #11) > > > I'm not sure I understand. I'm not suggesting to not bump the dependencies of the existing modules. I'm suggesting that you do not add the new modules, atk and at-spi2-atk. > > These new modules should likely go into jhbuild-optional.modules. ATK should be updated and at-spi2-atk could be added. This will allow people running distributions where these packages are difficult to maintain and easier way to install them. As I say in the changelog they are required by gtk 3.6, see all other build failures in EWS. (In reply to comment #11) > (From update of attachment 195078 [details]) > View in context: https://bugs.webkit.org/attachment.cgi?id=195078&action=review > > >>> Tools/gtk/jhbuild.modules:24 > >>> + <dep package="atk"/> > >> > >> The JHbuild moduleset is only for modules that need a specific version for the layout tests to pass. On the other hand, if a dependency is simply hard to get it should go into the optional moduleset. For instance, Cairo is there because even if you have a recent version of Cairo on your system, your version of Cairo might have rasterization differences from the version the bots are running. I've been very careful lately about adding new ones, since it adds a lot of overhead to compilation. > > > > So, how are people using internal jhbuild going to build wk if configure requires higher versions than the one provided by the jhbuild? > > I'm not sure I understand. I'm not suggesting to not bump the dependencies of the existing modules. I'm suggesting that you do not add the new modules, atk and at-spi2-atk. at-spi2-atk is already in the jhbuild.modules Created attachment 195504 [details]
Updated patch
Removed atk from the moduleset. Don't expect this to work on the bots unless a recent atk version is manually installed.
Comment on attachment 195504 [details]
Updated patch
Looks like the EWS lost the right output, but I guess it's related to not having the right ATK version. When landing this, do you mind also updated jhbuild-optional.modules so that people using the optional moduleset have the right version of ATK?
(In reply to comment #16) > (From update of attachment 195504 [details]) > Looks like the EWS lost the right output, but I guess it's related to not having the right ATK version. When landing this, do you mind also updated jhbuild-optional.modules so that people using the optional moduleset have the right version of ATK? Sure, but I still don't know how to land this without breaking all the bots. (In reply to comment #17) > (In reply to comment #16) > > (From update of attachment 195504 [details] [details]) > > Looks like the EWS lost the right output, but I guess it's related to not having the right ATK version. When landing this, do you mind also updated jhbuild-optional.modules so that people using the optional moduleset have the right version of ATK? > > Sure, but I still don't know how to land this without breaking all the bots. It's probably as simple as running ./Tools/jhbuild/jhbuild-wrapper --gtk buildone atk on all the bots that are missing ATK. (In reply to comment #16) > (From update of attachment 195504 [details]) > Looks like the EWS lost the right output, but I guess it's related to not having the right ATK version. When landing this, do you mind also updated jhbuild-optional.modules so that people using the optional moduleset have the right version of ATK? Surprisingly jhbuild-optional.modules, already has atk 2.5.3, so we don't need to do anything else. I still find very confusing having modules that are *required* to build webkitgtk+ in a moduleset called optional. Committed r147401: <http://trac.webkit.org/changeset/147401> (In reply to comment #18) > (In reply to comment #17) > > (In reply to comment #16) > > > (From update of attachment 195504 [details] [details] [details]) > > > Looks like the EWS lost the right output, but I guess it's related to not having the right ATK version. When landing this, do you mind also updated jhbuild-optional.modules so that people using the optional moduleset have the right version of ATK? > > > > Sure, but I still don't know how to land this without breaking all the bots. > > It's probably as simple as running ./Tools/jhbuild/jhbuild-wrapper --gtk buildone atk on all the bots that are missing ATK. This will backfire as the jhbuild is cleaned up on every change in module files, meaning that someone has to do this manually on each such occasion. Re-opened since this is blocked by bug 113767 (In reply to comment #19) > Surprisingly jhbuild-optional.modules, already has atk 2.5.3, so we don't need to do anything else. I still find very confusing having modules that are *required* to build webkitgtk+ in a moduleset called optional. We have never meant to use the WebKit jhbuild to provide dependencies. It's only used for a particular type of package that requires an exact version to pass layout tests. In the past, it was not uncommon to have hundreds of layout tests fail because the version of Cairo you had was too new. Even though your version of Cairo met the compile-time WebKit dependency requirements, you needed a specific version to pass tests. The jhbuild makes it easy and to be honest, even possible, to pass tests, since it records exactly what version the baselines rely on and provides the appropriate fonts. Before the jhbuild we didn't even know what packages affected layout test output. We're still figure one out now and then. The reason the jhbuild doesn't contain every dependency is that it's inefficient to compile everything twice if you don't absolutely need to. The other problem is that it is harder to distinguish normal dependencies from the "interesting" ones. By "interesting" I mean again the ones that require an exact version. The -optional moduleset appeared because time and again people wanted to add dependencies to the jhbuild that aren't "interesting," but merely because those dependencies were not not in their distributions package manager. I originally resisted adding these to jhbuild, since instructions for installing dependencies were listed on the trac wiki. At some point I realized I was wrong though, since it should be as easy as possible to compile WebKit. By separating these hard-to-get dependencies into -optional, we still avoid making everyone compile everything twice and know which dependencies are "interesting." It's possible that at the current moment the main jhbuild moduleset contains dependencies that it shouldn't. Those should be removed where appropriate. Hopefully this comment is helpful. Gustavo and I did not mean the jhbuild to be confusing and infuriating, we meant it to be helpful and to allow something that was just about impossible in the past. (In reply to comment #21) > > It's probably as simple as running ./Tools/jhbuild/jhbuild-wrapper --gtk buildone atk on all the bots that are missing ATK. > > This will backfire as the jhbuild is cleaned up on every change in module files, meaning that someone has to do this manually on each such occasion. This is unfortunate. Perhaps optional jhbuild modules should go into another root. And to be clear the issue here is that you want to depend on a very new version of ATK that the bots don't have. The fact is that the jhbuild wasn't meant as a solution to that problem. Perhaps the answer is to rely on a less recent version of ATK. The bots are a pretty good litmus test for what a reasonable dependency is. That they don't have that version likely means that this change is going to cause headaches for many people. (In reply to comment #23) > (In reply to comment #19) > > Surprisingly jhbuild-optional.modules, already has atk 2.5.3, so we don't need to do anything else. I still find very confusing having modules that are *required* to build webkitgtk+ in a moduleset called optional. > > We have never meant to use the WebKit jhbuild to provide dependencies. It's only used for a particular type of package that requires an exact version to pass layout tests. In the past, it was not uncommon to have hundreds of layout tests fail because the version of Cairo you had was too new. Even though your version of Cairo met the compile-time WebKit dependency requirements, you needed a specific version to pass tests. The jhbuild makes it easy and to be honest, even possible, to pass tests, since it records exactly what version the baselines rely on and provides the appropriate fonts. Before the jhbuild we didn't even know what packages affected layout test output. We're still figure one out now and then. I understand what jhbuild is for and it was a great idea, but the fact is that bots use it to build, so if you introduce a new dependency that happens to depend on another lib that is not "interesting" it doesn't build. > The reason the jhbuild doesn't contain every dependency is that it's inefficient to compile everything twice if you don't absolutely need to. The other problem is that it is harder to distinguish normal dependencies from the "interesting" ones. By "interesting" I mean again the ones that require an exact version. The problem is when a "normal" dependency requires another not "interesting". > The -optional moduleset appeared because time and again people wanted to add dependencies to the jhbuild that aren't "interesting," but merely because those dependencies were not not in their distributions package manager. I originally resisted adding these to jhbuild, since instructions for installing dependencies were listed on the trac wiki. At some point I realized I was wrong though, since it should be as easy as possible to compile WebKit. By separating these hard-to-get dependencies into -optional, we still avoid making everyone compile everything twice and know which dependencies are "interesting." This is because the instructions to build webkit now relies on the internal jhbuild, even for people who will never run the tests. > It's possible that at the current moment the main jhbuild moduleset contains dependencies that it shouldn't. Those should be removed where appropriate. Hopefully this comment is helpful. Gustavo and I did not mean the jhbuild to be confusing and infuriating, we meant it to be helpful and to allow something that was just about impossible in the past. I insist the jhbuild is not the problem, it was a great idea, but if it's only meant for bots and people wanting to run the tests, then the main build should not depend on it. I understand that jhbuild is meant for the bots, but you need to build in the end, and now it's not possible to build with the dependencies we want. (In reply to comment #25) > And to be clear the issue here is that you want to depend on a very new version of ATK that the bots don't have. The fact is that the jhbuild wasn't meant as a solution to that problem. Perhaps the answer is to rely on a less recent version of ATK. The bots are a pretty good litmus test for what a reasonable dependency is. That they don't have that version likely means that this change is going to cause headaches for many people. It's not that *I* want to depend on a new ATK, but that *we* want to bump the GTK+ requirements, as we agreed, and the new GTK version requires a newer ATK version. We already depend on things that could be optional like libsecret which causes headaches to many people, but unfortunately ATK can't be optional. (In reply to comment #26) > This is because the instructions to build webkit now relies on the internal jhbuild, even for people who will never run the tests. The assumption is that everyone developing WebKit will run tests. If you are building the tarball for installation on your system, you should not use the jhbuild. > I understand that jhbuild is meant for the bots, but you need to build in the end, and now it's not possible to build with the dependencies we want. I think we should consider a less aggressive dependency bump. This is clearly going to cause pain. (In reply to comment #28) > (In reply to comment #26) > > > This is because the instructions to build webkit now relies on the internal jhbuild, even for people who will never run the tests. > > The assumption is that everyone developing WebKit will run tests. If you are building the tarball for installation on your system, you should not use the jhbuild. > > > I understand that jhbuild is meant for the bots, but you need to build in the end, and now it's not possible to build with the dependencies we want. > > I think we should consider a less aggressive dependency bump. This is clearly going to cause pain. I don't see the pain, we just need to make the bots build atk, it's a very small module that builds quite fast, I don't think it affects build times that much. (In reply to comment #24) > (In reply to comment #21) > > > > It's probably as simple as running ./Tools/jhbuild/jhbuild-wrapper --gtk buildone atk on all the bots that are missing ATK. > > > > This will backfire as the jhbuild is cleaned up on every change in module files, meaning that someone has to do this manually on each such occasion. > > This is unfortunate. Perhaps optional jhbuild modules should go into another root. I'd rather not introduce another root. An option would be to introduce a webkitgtk-building-dependencies metamodule and have that referenced in WEBKIT_EXTRA_MODULES when necessary (i.e. in the builders). While this slightly increases the complexity it keeps the solution inside the jhbuild bounds. Another option is to switch the builders to a more up-to-date distribution (unlike the Debian unstable) and recommend that as a sort of a developer platform, though I'm not sure only the builders are affected in this case. (In reply to comment #30) > (In reply to comment #24) > > (In reply to comment #21) > > > > > > It's probably as simple as running ./Tools/jhbuild/jhbuild-wrapper --gtk buildone atk on all the bots that are missing ATK. > > > > > > This will backfire as the jhbuild is cleaned up on every change in module files, meaning that someone has to do this manually on each such occasion. > > > > This is unfortunate. Perhaps optional jhbuild modules should go into another root. > > I'd rather not introduce another root. > > An option would be to introduce a webkitgtk-building-dependencies metamodule and have that referenced in WEBKIT_EXTRA_MODULES when necessary (i.e. in the builders). While this slightly increases the complexity it keeps the solution inside the jhbuild bounds. > > Another option is to switch the builders to a more up-to-date distribution (unlike the Debian unstable) and recommend that as a sort of a developer platform, though I'm not sure only the builders are affected in this case. Our bots have been updated now, so let's try again. Gustavo, could you update your bots too? The 32 bit bot seems to be broken anyway, no? Created attachment 196330 [details]
Patch for landing
(In reply to comment #31) > Our bots have been updated now, so let's try again. Gustavo, could you update your bots too? The 32 bit bot seems to be broken anyway, no? I'm updating my EWS bots and 32 bits. It is indeed broken, seems to be caused by being under a lot of stress, I asked the collabora sysadmins to take a look, thanks! (In reply to comment #33) > (In reply to comment #31) > > Our bots have been updated now, so let's try again. Gustavo, could you update your bots too? The 32 bit bot seems to be broken anyway, no? > > I'm updating my EWS bots and 32 bits. It is indeed broken, seems to be caused by being under a lot of stress, I asked the collabora sysadmins to take a look, thanks! Thanks! Joanie informed me that a specific version of ATK (greater than the one we depend on) is now required to pass layout tests. Mea culpa. :( Feel free to land your original patch. Comment on attachment 196330 [details] Patch for landing Clearing flags on attachment: 196330 Committed r147547: <http://trac.webkit.org/changeset/147547> All reviewed patches have been landed. Closing bug. |