Bug 192702 - [Flatpak] Second run update-webkitwpe-flatpak fails
Summary: [Flatpak] Second run update-webkitwpe-flatpak fails
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Philippe Normand
URL:
Keywords: InRadar
Depends on:
Blocks: 191196
  Show dependency treegraph
 
Reported: 2018-12-14 09:14 PST by Adrian Perez
Modified: 2019-04-08 05:15 PDT (History)
7 users (show)

See Also:


Attachments
Patch (2.93 KB, patch)
2019-04-08 02:55 PDT, Philippe Normand
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Adrian Perez 2018-12-14 09:14:32 PST
Steps to reproduce:

 1. git clean -xdff
 2. ./Tools/Scripts/update-webkitwpe-flatpak
 3. ./Tools/Scripts/update-webkitwpe-flatpak

The first invocation of “update-webkitwpe-flatpak” builds and install
everything correctly. Subsequent invocations always fail (command output
below). The same goes for “update-webkitgtk-flatpak”.


-----

% update-webkitwpe-flatpak
Updating Flatpak environment for WPE (Release)
Looking for updates...
Updating in user:
org.gnome.Platform/x86_64/3.28 flathub 9d3fe926d425
Updating for user: org.gnome.Platform/x86_64/3.28 from flathub
Error: Failed to update org.gnome.Platform/x86_64/3.28: Commit 9d3fe926d425ad837e40a0945020a00f09c1f531335987a753e07f276d76ee36: GPG verification enabled, but no signatures found (use gpg-verify=false in remote config to disable)
error: There were one or more errors
Traceback (most recent call last):
  File "/home/aperez/WebKit/Tools/Scripts/update-webkitwpe-flatpak", line 28, in <module>
    WebkitFlatpak.load_from_args(["--wpe", "--update"] + sys.argv[1:]).run()
  File "/home/aperez/WebKit/Tools/flatpak/flatpakutils.py", line 774, in run
    self.update_all()
  File "/home/aperez/WebKit/Tools/flatpak/flatpakutils.py", line 888, in update_all
    m.update()
  File "/home/aperez/WebKit/Tools/flatpak/flatpakutils.py", line 441, in update
    *extra_args, comment=comment)
  File "/home/aperez/WebKit/Tools/flatpak/flatpakutils.py", line 245, in flatpak
    return subprocess.check_call(command)
  File "/usr/lib64/python2.7/subprocess.py", line 190, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['flatpak', 'update', '--user', 'org.gnome.Platform', '3.28', '--commit', '9d3fe926d425ad837e40a0945020a00f09c1f531335987a753e07f276d76ee36', '--assumeyes']' returned non-zero exit status 1
Comment 1 Philippe Normand 2019-03-23 07:31:18 PDT
The error is different now, the issue is that we try to update the runtime and sdk to a commit that no longer exists in the flathub repo.
Comment 2 Philippe Normand 2019-03-23 07:38:28 PDT
Another issue, the install/update logic is flawed, the sdk/runtime are installed, the latest version published on flathub are installed, but i think we should instead install the versions specified by hashes from the manifest. Currently the hashes are used only for the update workflow, which is prone to errors because the flathub admins seem to periodically remove old commits from their OStrees.
Comment 3 Philippe Normand 2019-03-23 07:44:21 PDT
Today's hashes:

diff --git a/Tools/flatpak/org.webkit.WebKit.yaml b/Tools/flatpak/org.webkit.WebKit.yaml
index fe5b008f4be..167e43bc670 100644
--- a/Tools/flatpak/org.webkit.WebKit.yaml
+++ b/Tools/flatpak/org.webkit.WebKit.yaml
@@ -2,8 +2,8 @@ app-id: %(PORTNAME)s
 runtime: org.gnome.Platform
 runtime-version: "3.28"
 # Control the exact version of the Sdk/Runtime that is being used.
-sdk-hash: ba3017972f1cc6c7c8b0ed7f80d3297ad4c6f4076bd799d0bab0e2cbaa3e3c0a
-runtime-hash: 9d3fe926d425ad837e40a0945020a00f09c1f531335987a753e07f276d76ee36
+sdk-hash: 31af0c10f8e42151220a728b52df0a5c37ea8e4b033825f40c30e44d11cb31d1
+runtime-hash: 6d1d0ebbd72404c61d109307eb2240542b7ad82608bc6428bba6f3eebcfc8bf3
 sdk: org.gnome.Sdk
 command: %(COMMAND)s
 finish-args:
Comment 4 Philippe Normand 2019-03-23 07:46:59 PDT
I would remove the hashes, to be honest. Unless we can have guarantees the server admins that old commits are kept.

Any thoughts?
Comment 5 Michael Catanzaro 2019-03-24 07:52:42 PDT
(In reply to Philippe Normand from comment #2)
> Another issue, the install/update logic is flawed, the sdk/runtime are
> installed, the latest version published on flathub are installed, but i
> think we should instead install the versions specified by hashes from the
> manifest. Currently the hashes are used only for the update workflow

Ow, that's bad. :/

(In reply to Philippe Normand from comment #4)
> I would remove the hashes, to be honest. Unless we can have guarantees the
> server admins that old commits are kept.
> 
> Any thoughts?

We do have such guarantees. Alex has promised to keep old commits for some period of time (at least two months; I think I even heard six months? I don't remember the exact time guarantee). But if we leave old hashes there indefinitely, like we've been doing, then yes they will eventually break. Like anything, they have to be maintained.

If you remove the hashes, then we'll all have different development environments and won't be able to have reproducible tests, so we'll never be able to get rid of the jhbuild.
Comment 6 Philippe Normand 2019-03-24 08:33:45 PDT
(In reply to Michael Catanzaro from comment #5)
> If you remove the hashes, then we'll all have different development
> environments and won't be able to have reproducible tests, so we'll never be
> able to get rid of the jhbuild.

The install/update logic needs to be reversed then, as explained in comment 2
Comment 7 Philippe Normand 2019-04-08 02:55:23 PDT
Created attachment 366924 [details]
Patch
Comment 8 WebKit Commit Bot 2019-04-08 05:14:49 PDT
Comment on attachment 366924 [details]
Patch

Clearing flags on attachment: 366924

Committed r243999: <https://trac.webkit.org/changeset/243999>
Comment 9 WebKit Commit Bot 2019-04-08 05:14:50 PDT
All reviewed patches have been landed.  Closing bug.
Comment 10 Radar WebKit Bug Importer 2019-04-08 05:15:27 PDT
<rdar://problem/49693050>