Bug 187242

Summary: flatpak-builder fails to clone repositories from https://anongit.freedesktop.org/
Product: WebKit Reporter: Fujii Hironori <Hironori.Fujii>
Component: WebKitGTKAssignee: Nobody <webkit-unassigned>
Status: RESOLVED INVALID    
Severity: Normal CC: bugs-noreply, calvaris, mcatanzaro, pnormand, tsaunier
Priority: P2    
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   

Description Fujii Hironori 2018-07-01 22:25:10 PDT
flatpak-builder fails to clone recursive repository behind proxy

> $ python3 ./Tools/Scripts/update-webkitgtk-flatpak
> Updating Flatpak environment for GTK (Release)
> Looking for updates...
> Looking for updates...
> Building org.webkit.GTK and dependencies in /home/fujii/work/webkit/ga/WebKitBuild/GTK/FlatpakTreeRelease
> Downloading sources
> Fetching git repo https://github.com/apache/httpd.git, ref refs/tags/2.4.33
> remote: Counting objects: 1, done.
> remote: Total 1 (delta 0), reused 1 (delta 0), pack-reused 0
> Unpacking objects: 100% (1/1), done.
> Fetching git repo https://github.com/libevent/libevent.git, ref refs/tags/release-2.1.8-stable
> remote: Counting objects: 1, done.
> remote: Total 1 (delta 0), reused 1 (delta 0), pack-reused 0
> Unpacking objects: 100% (1/1), done.
> Fetching git repo https://chromium.googlesource.com/webm/libvpx, ref refs/tags/v1.7.0
> remote: Counting objects: 1, done
> remote: Finding sources: 100% (1/1)
> remote: Total 1 (delta 0), reused 1 (delta 0)
> Unpacking objects: 100% (1/1), done.
> fatal: unable to access 'https://anongit.freedesktop.org/git/gstreamer/gstreamer/': Received HTTP code 503 from proxy after CONNECT
> Failed to download sources: module gstreamer: Child process exited with code 128
> Traceback (most recent call last):
>   File "./Tools/Scripts/update-webkitgtk-flatpak", line 28, in <module>
>     WebkitFlatpak.load_from_args(["--gtk", "--update"] + sys.argv[1:]).run()
>   File "./Tools/flatpak/flatpakutils.py", line 711, in run
>     return self.setup_dev_env()
>   File "./Tools/flatpak/flatpakutils.py", line 738, in setup_dev_env
>     subprocess.check_call(builder_args)
>   File "/usr/lib/python3.6/subprocess.py", line 291, in check_call
>     raise CalledProcessError(retcode, cmd)
> subprocess.CalledProcessError: Command '['flatpak-builder', '--disable-rofiles-fuse', '--state-dir', '/home/fujii/work/webkit/ga/WebKitBuild/FlatpakCache', '--ccache', '/home/fujii/work/webkit/ga/WebKitBuild/GTK/FlatpakTreeRelease', '--force-clean', '/home/fujii/work/webkit/ga/WebKitBuild/FlatpakCache/org.webkit.GTK-generated.json', '--build-only', '--stop-at=org.webkit.GTK']' returned non-zero exit status 1.
> $

Gstreamer and Gstreamer plugins have 'common' as a submodule.
Comment 1 Fujii Hironori 2018-07-01 23:52:04 PDT
I was wrong. This is not a problm flatpak-builder.
Cloning repositories from https://anongit.freedesktop.org/ is unstable.
Comment 2 Philippe Normand 2018-07-04 02:34:33 PDT
I think we should rely on tarballs whenever possible. Cloning over git takes more time and is less reliable and less proxy-friendly.
Comment 3 Michael Catanzaro 2018-07-04 05:52:26 PDT
Actually, flatpak-builder does shallow clones, so they should be fast, and Thibault is using the https:// protocol here. (I agree that we should not use git:// anywhere.) So I think git is fine.

Anyway, tarballs are fine too.
Comment 4 Thibault Saunier 2018-07-04 05:57:39 PDT
(In reply to Philippe Normand from comment #2)
> I think we should rely on tarballs whenever possible. Cloning over git takes
> more time and is less reliable and less proxy-friendly.

As Michael said, we are doing shallow clones here, so the size should be very similar. Using git repos makes it simpler to update (only change the tag and you are done) so I am using git on purpose.

(In reply to Fujii Hironori from comment #1)
> I was wrong. This is not a problm flatpak-builder.
> Cloning repositories from https://anongit.freedesktop.org/ is unstable.

Let's close the bug then? I hope freedesktop get back to normal functioning but it is a bit out of ends.
Comment 5 Michael Catanzaro 2018-07-04 06:29:15 PDT
(In reply to Thibault Saunier from comment #4)
> Let's close the bug then? I hope freedesktop get back to normal functioning
> but it is a bit out of ends.

No. If the code hosting is unstable, then we can't depend on it.

Surely the next step for the flatpak support is to work with Adrian to switch the bots from using jhbuild to flatpak. And then we will finally have consistent test results, can drop the jhbuild support, and can reenable tests on our EWS. None of that will be possible if the servers we rely on are unreliable.

So if there are still problems with anongit.freedesktop.org, I would consider tarballs indeed. Or mirroring the git repo elsewhere.

For GNOME, I have to host a few tarballs myself on people.gnome.org, since it's required for builds to be reliable. We can be happy that we probably won't need to do that.
Comment 6 Thibault Saunier 2018-07-04 06:43:04 PDT
(In reply to Michael Catanzaro from comment #5)
> (In reply to Thibault Saunier from comment #4)
> > Let's close the bug then? I hope freedesktop get back to normal functioning
> > but it is a bit out of ends.
> 
> No. If the code hosting is unstable, then we can't depend on it.

Well the tarball are on the same server, so I would expect more or less the same instabilities, it really doesn't happen often.
 
> Surely the next step for the flatpak support is to work with Adrian to
> switch the bots from using jhbuild to flatpak. And then we will finally have
> consistent test results, can drop the jhbuild support, and can reenable
> tests on our EWS. None of that will be possible if the servers we rely on
> are unreliable.
> 
> So if there are still problems with anongit.freedesktop.org, I would
> consider tarballs indeed. Or mirroring the git repo elsewhere.

We could use the official github mirror too. I hope and expect the gitlab server to work properly once we move to it also (should happen soon I believe).
Comment 7 Fujii Hironori 2018-07-04 18:13:36 PDT
I tried update-webkitgtk-flatpak today, but failed with the same error again.
I tried to invoke "git ls-remote https://anongit.freedesktop.org/git/gstreamer/common.git" 10 times.

> $ for i in $(seq 10); do echo -n "try $i: "; git ls-remote https://anongit.freedesktop.org/git/gstreamer/common.git | wc;done
> try 1:     258     516   18047
> try 2:     258     516   18047
> try 3:     258     516   18047
> try 4: fatal: unable to access 'https://anongit.freedesktop.org/git/gstreamer/common.git/': Received HTTP code 503 from proxy after CONNECT
>       0       0       0
> try 5:     258     516   18047
> try 6: fatal: unable to access 'https://anongit.freedesktop.org/git/gstreamer/common.git/': Received HTTP code 503 from proxy after CONNECT
>       0       0       0
> try 7:     258     516   18047
> try 8:     258     516   18047
> try 9:     258     516   18047
> try 10:     258     516   18047
> $

Yes, I'm using a proxy. This seems a problem of the proxy if other doesn't see this.
Comment 8 Fujii Hironori 2018-07-04 18:28:11 PDT
Umm, I guess the server randomly refuses connections.

> $ curl --head --silent https://anongit.freedesktop.org/git/gstreamer/common.git/
> HTTP/1.1 503 Service Unavailable
> Cache-Control: no-cache
> Pragma: no-cache
> Content-Type: text/html; charset=utf-8
> Proxy-Connection: close
> Connection: close
> Content-Length: 787
> 
> $
Comment 9 Fujii Hironori 2018-07-04 20:41:42 PDT
> $ curl --head --silent https://gstreamer.freedesktop.org/src/gstreamer/gstreamer-1.14.1.tar.xz
> HTTP/1.1 503 Service Unavailable
> Cache-Control: no-cache
> Pragma: no-cache
> Content-Type: text/html; charset=utf-8
> Proxy-Connection: close
> Connection: close
> Content-Length: 1274
> 
> $

Downloading tarballs also fails sometimes.
But, this hasn't been a problem since jhbuild caches tarballs in ~/.cache/jhbuild/downloads/.
Comment 10 Thibault Saunier 2018-07-05 04:57:29 PDT
(In reply to Fujii Hironori from comment #9)
> > $ curl --head --silent https://gstreamer.freedesktop.org/src/gstreamer/gstreamer-1.14.1.tar.xz
> > HTTP/1.1 503 Service Unavailable
> > Cache-Control: no-cache
> > Pragma: no-cache
> > Content-Type: text/html; charset=utf-8
> > Proxy-Connection: close
> > Connection: close
> > Content-Length: 1274
> > 
> > $
> 
> Downloading tarballs also fails sometimes.
> But, this hasn't been a problem since jhbuild caches tarballs in
> ~/.cache/jhbuild/downloads/.

Flatpak-builder also caches everything in WebKitBuild/FlatpakCache.
Comment 11 Fujii Hironori 2018-07-05 06:20:23 PDT
(In reply to Thibault Saunier from comment #10)
> Flatpak-builder also caches everything in WebKitBuild/FlatpakCache.

Right.
In my comment #0, git is fetching repository only the diff. But, it failed because it tried to connect the server.

> remote: Counting objects: 1, done.
> remote: Total 1 (delta 0), reused 1 (delta 0), pack-reused 0
Comment 12 Fujii Hironori 2018-07-05 06:22:30 PDT
I tested in my home and Goolge Clould Platform too.
Both have no problem on connecting the server.
This problem happens only from my office. Umm.

> for i in $(seq 10); do echo try $i: ; curl --head --silent https://anongit.freedesktop.org/git/gstreamer/common.git/; done
Comment 13 Fujii Hironori 2018-07-06 03:43:40 PDT
I'm very sorry. This is a problem completely in my side.
I can solve this problem by changing proxy.

My company has three types of proxy, ipv6-only, ipv4-only, ipv6-prefer proxies.
I was using ipv6-prefer proxy. This is the problem.

Closed as INVALID.