Bug 191196

Summary: [WPE][GTK] Remove either the jhbuild or the flatpak environment
Product: WebKit Reporter: Michael Catanzaro <mcatanzaro>
Component: WebKitGTKAssignee: Michael Catanzaro <mcatanzaro>
Status: RESOLVED INVALID    
Severity: Normal CC: aperez, bugs-noreply, Hironori.Fujii, mcatanzaro, pnormand, tsaunier
Priority: P2    
Version: WebKit Nightly Build   
Hardware: PC   
OS: Linux   
Bug Depends on: 188654, 190241, 192702    
Bug Blocks:    
Attachments:
Description Flags
Patch pnormand: review-

Description Michael Catanzaro 2018-11-02 09:26:35 PDT
Having two separate build environments has gone on for far too long. We need to either switch the bots to using webkit-flatpak and remove the jhbuild environment, or else remove webkit-flatpak. At this point I no longer have a strong preference either way.
Comment 1 Michael Catanzaro 2018-11-02 10:17:39 PDT
Adrian has reminded me that we prefer to switch to flatpak so that we all have exactly the same build environment, and also reminded me that it should be possible to build the sysroot using flatpak but execute our tests outside it, so we can test WebKit's sandbox rather than the flatpak sandbox. So flatpak would be used for building, but not running.

Still, the project seems stalled, the build rules have become desynced from jhbuild due to patches being added in jhbuild but not to flatpak, and we have unresolved bugs like bug #190241 and bug #188654, so the current state is not good and the transition period has been quite long already.
Comment 2 Michael Catanzaro 2018-12-15 09:56:34 PST
(In reply to Michael Catanzaro from comment #1)
> Still, the project seems stalled, the build rules have become desynced from
> jhbuild due to patches being added in jhbuild but not to flatpak, and we
> have unresolved bugs like bug #190241 and bug #188654, so the current state
> is not good and the transition period has been quite long already.

Adrian, can you remind us what other issues block the switch to flatpak? I think you had something else you wanted fixed before we remove the jhbuild, right?
Comment 3 Adrian Perez 2018-12-15 15:37:57 PST
(In reply to Michael Catanzaro from comment #2)
> (In reply to Michael Catanzaro from comment #1)
> > Still, the project seems stalled, the build rules have become desynced from
> > jhbuild due to patches being added in jhbuild but not to flatpak, and we
> > have unresolved bugs like bug #190241 and bug #188654, so the current state
> > is not good and the transition period has been quite long already.
> 
> Adrian, can you remind us what other issues block the switch to flatpak? I
> think you had something else you wanted fixed before we remove the jhbuild,
> right?

On top of the two issues, you mentioned:

 - Bug #192702: If you run update-webkit{gtk,wpe}-flatpak more than once,
   all runs after the first one fail.

 - Figuring out a way to use the Flatpak “root” file system as a sysroot
   for building and running tests, but *without* using the Flatpak sandbox.
   The motivation is twofold: in one hand, this would allow actualy testing
   the Bubblewrap-based sandbox; and on the other hand this would allow
   running bots on containers (as we are already doing now).
Comment 4 Adrian Perez 2018-12-15 15:39:21 PST
JFTR, for the desync between build rules between JHBuild and Flatpak,
reviewers should r- any patch that touches one but not the others.
Whether we enforce keeping them in sync or not, is completely up to
us ;-)
Comment 5 Michael Catanzaro 2019-01-01 10:22:59 PST
(In reply to Adrian Perez from comment #4)
> JFTR, for the desync between build rules between JHBuild and Flatpak,
> reviewers should r- any patch that touches one but not the others.
> Whether we enforce keeping them in sync or not, is completely up to
> us ;-)

I had considered doing this several times as people have proposed several patches updating only one and not the other. GStreamer, in particular, has become desynced between the two environments. But ultimately I decided the flatpak environment is secondary and allowed to become stale until it is ready to replace the jhbuild.

(In reply to Adrian Perez from comment #3)
> On top of the two issues, you mentioned:
> 
>  - Bug #192702: If you run update-webkit{gtk,wpe}-flatpak more than once,
>    all runs after the first one fail.
> 
>  - Figuring out a way to use the Flatpak “root” file system as a sysroot
>    for building and running tests, but *without* using the Flatpak sandbox.
>    The motivation is twofold: in one hand, this would allow actualy testing
>    the Bubblewrap-based sandbox; and on the other hand this would allow
>    running bots on containers (as we are already doing now).

OK, I agree these are both important. We currently do not have a concrete plan to work on them. It's very important to create such a plan ASAP.

Hopefully such a plan is created very soon. Otherwise, I think it's really time to remove the flatpak environment until such time it's ready to be relanded, because:

 * There has been no progress on this bug in the two months since I reported it; and
 * The flatpak environment has been unusable for several weeks now without anyone seeming to care, bug #190241
Comment 6 Michael Catanzaro 2019-06-10 07:56:18 PDT
(In reply to Michael Catanzaro from comment #0)
> Having two separate build environments has gone on for far too long.

This was half a year ago, and it's not OK. Let's give up for now and try again later when we're ready to finish it.
Comment 7 Michael Catanzaro 2019-06-10 07:56:23 PDT
Created attachment 371743 [details]
Patch
Comment 8 Philippe Normand 2019-06-10 08:36:00 PDT
Comment on attachment 371743 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=371743&action=review

> Tools/ChangeLog:12
> +        linger in WebKit for excessive periods of time. The point of the transition to flatpak was
> +        to remove the jhbuild environment, but, at least for the time being, this has clearly

I disagree. The point (for me anyway) was to provide a modern alternative to jhbuild, and eventually replace it in the long run. It doesn't mean we can't have both systems side-by-side meanwhile. The maintenance cost of the flatpak build is low anyway. Removing it and resurrecting it later would be more work.
Comment 9 Adrian Perez 2019-06-11 01:13:27 PDT
(In reply to Philippe Normand from comment #8)
> Comment on attachment 371743 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=371743&action=review
> 
> > Tools/ChangeLog:12
> > +        linger in WebKit for excessive periods of time. The point of the transition to flatpak was
> > +        to remove the jhbuild environment, but, at least for the time being, this has clearly
> 
> I disagree. The point (for me anyway) was to provide a modern alternative to
> jhbuild, and eventually replace it in the long run. It doesn't mean we can't
> have both systems side-by-side meanwhile. The maintenance cost of the
> flatpak build is low anyway. Removing it and resurrecting it later would be
> more work.

I basically agree with Philippe. The cost of maintaining the Flatpak
environment infrastructure has been negligible, and it brings in more
convenience than pain — I would rather keep it.
Comment 10 Philippe Normand 2020-03-18 03:17:28 PDT
Comment on attachment 371743 [details]
Patch

The flatpak infra will be ditched if the experiment with the new SDK fails.