Bug 230211 - [webkitpy] AutoInstall broken on Linux
Summary: [webkitpy] AutoInstall broken on Linux
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Jonathan Bedard
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-09-13 02:21 PDT by Philippe Normand
Modified: 2021-09-21 11:14 PDT (History)
5 users (show)

See Also:


Attachments
Move Linux special case to webkitpy (3.81 KB, patch)
2021-09-20 15:44 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Add root CA (5.49 KB, patch)
2021-09-20 16:00 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Philippe Normand 2021-09-13 02:21:55 PDT
The issue:

Downloading setuptools-56.2.0...
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)>
Failed to download setuptools, retrying
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)>
Failed to download setuptools, retrying
Failed to install setuptools-56.2.0!
Traceback (most recent call last):
  File "Tools/Scripts/git-webkit", line 28, in <module>
    from webkitpy.common.config.committers import CommitterList
  File "/home/buildbot/igalia-jsc32-armv7-ews/JSC-ARMv7-32bits-Build-EWS/build/Tools/Scripts/webkitpy/__init__.py", line 78, in <module>
    import webkitscmpy
  File "/home/buildbot/igalia-jsc32-armv7-ews/JSC-ARMv7-32bits-Build-EWS/build/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/__init__.py", line 57, in <module>
    from webkitscmpy.commit import Commit
  File "/home/buildbot/igalia-jsc32-armv7-ews/JSC-ARMv7-32bits-Build-EWS/build/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/commit.py", line 24, in <module>
    import six
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 963, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 906, in _find_spec
  File "/home/buildbot/igalia-jsc32-armv7-ews/JSC-ARMv7-32bits-Build-EWS/build/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 602, in find_spec
    loader = cls.find_module(fullname, path=path)
  File "/home/buildbot/igalia-jsc32-armv7-ews/JSC-ARMv7-32bits-Build-EWS/build/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 616, in find_module
    cls.install(name)
  File "/home/buildbot/igalia-jsc32-armv7-ews/JSC-ARMv7-32bits-Build-EWS/build/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 591, in install
    return all([to_install.install() for to_install in packages])
  File "/home/buildbot/igalia-jsc32-armv7-ews/JSC-ARMv7-32bits-Build-EWS/build/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 591, in <listcomp>
    return all([to_install.install() for to_install in packages])
  File "/home/buildbot/igalia-jsc32-armv7-ews/JSC-ARMv7-32bits-Build-EWS/build/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 273, in install
    AutoInstall.install('setuptools')
  File "/home/buildbot/igalia-jsc32-armv7-ews/JSC-ARMv7-32bits-Build-EWS/build/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 591, in install
    return all([to_install.install() for to_install in packages])
  File "/home/buildbot/igalia-jsc32-armv7-ews/JSC-ARMv7-32bits-Build-EWS/build/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 591, in <listcomp>
    return all([to_install.install() for to_install in packages])
  File "/home/buildbot/igalia-jsc32-armv7-ews/JSC-ARMv7-32bits-Build-EWS/build/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 296, in install
    archive.unpack(temp_location)
  File "/home/buildbot/igalia-jsc32-armv7-ews/JSC-ARMv7-32bits-Build-EWS/build/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 126, in unpack
    raise IOError('Failed to find archive at {}'.format(self.path))
OSError: Failed to find archive at /home/buildbot/igalia-jsc32-armv7-ews/JSC-ARMv7-32bits-Build-EWS/build/Tools/Scripts/libraries/autoinstalled/python-3/setuptools-56.2.tar.gz
Comment 1 Philippe Normand 2021-09-13 02:27:26 PDT
I landed r282322 as an emergency fix for EWS/etc bots. I am not sure why this custom cacert is needed in the first place.

Is it due to a Python limitation on Apple platforms? In linux we can usually assume the host has valid root certificates, otherwise, maybe we could bootstrap 'certifi' before installing setuptools? That would likely avoid the need for a custom cert in the first place.
Comment 2 Jonathan Bedard 2021-09-13 07:23:37 PDT
A large number of Apple's bots rely on a different pypi source for their packages, and we have a bit of a chicken-egg problem with getting that cacert. Basically, the cacert in webkitcorepy isn't just for pypi.org.

Will get it updated, this is the cost of owning the cacert....
Comment 3 Michael Catanzaro 2021-09-15 13:00:28 PDT
Might want to rename the file to avoid confusion with http://www.cacert.org/ ;)
Comment 4 Radar WebKit Bug Importer 2021-09-20 02:22:17 PDT
<rdar://problem/83298684>
Comment 5 Jonathan Bedard 2021-09-20 15:44:35 PDT
Created attachment 438742 [details]
Move Linux special case to webkitpy
Comment 6 Jonathan Bedard 2021-09-20 16:00:00 PDT
Created attachment 438746 [details]
Add root CA
Comment 7 Jonathan Bedard 2021-09-20 16:02:44 PDT
I think we'd prefer the second one, but if we aren't going to do that, we should do the first so that the Linux special case is in webkitpy and doesn't apply to other callers of webkitpy
Comment 8 Philippe Normand 2021-09-21 11:14:25 PDT
I'll try to test tomorrow.