Bug 223677 – Autoinstalling keyring-7.3.1 is failing with Python 2

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Fujii Hironori]

$ git clean -dfx
$ Tools/Scripts/webkit-patch --help
Downloading setuptools-44.1.1...                 
Installing setuptools-44.1.1...
Installed setuptools-44.1.1!
Downloading wheel-0.35.1...
Installing wheel-0.35.1...
Installed wheel-0.35.1!
Downloading six-1.15.0...
Installing six-1.15.0...
Installed six-1.15.0!
Downloading whichcraft-0.6.1...
Installing whichcraft-0.6.1...
Installed whichcraft-0.6.1!
Downloading fasteners-0.15.0...
Installing fasteners-0.15.0...
Installed fasteners-0.15.0!
Downloading monotonic-1.5.0...
Installing monotonic-1.5.0...
Installed monotonic-1.5.0!
Downloading requests-2.24.0...
Installing requests-2.24.0...
Installed requests-2.24.0!
Downloading urllib3-1.25.10...
Installing urllib3-1.25.10...
Installed urllib3-1.25.10!
Downloading chardet-3.0.4...
Installing chardet-3.0.4...
Installed chardet-3.0.4!
Downloading certifi-2020.6.20...
Installing certifi-2020.6.20...
Installed certifi-2020.6.20!
Downloading idna-2.10.0...
Installing idna-2.10.0...
Installed idna-2.10.0!
Downloading PySocks-1.7.1...
Installing PySocks-1.7.1...
Installed PySocks-1.7.1!
Downloading xmltodict-0.11.0...
Installing xmltodict-0.11.0...
Installed xmltodict-0.11.0!
Traceback (most recent call last):
  File "Tools/Scripts/webkit-patch", line 42, in <module>
    from webkitpy.tool.main import WebKitPatch
  File "/mnt/c/home/webkit/ga/Tools/Scripts/webkitpy/tool/main.py", line 40, in <module>
    from webkitpy.tool import commands
  File "/mnt/c/home/webkit/ga/Tools/Scripts/webkitpy/tool/commands/__init__.py", line 4, in <module>
    from webkitpy.tool.commands.analyzechangelog import AnalyzeChangeLog
  File "/mnt/c/home/webkit/ga/Tools/Scripts/webkitpy/tool/commands/analyzechangelog.py", line 38, in <module>
    from webkitpy.tool import steps
  File "/mnt/c/home/webkit/ga/Tools/Scripts/webkitpy/tool/steps/__init__.py", line 56, in <module>
    from webkitpy.tool.steps.postdiffforrevert import PostDiffForRevert
  File "/mnt/c/home/webkit/ga/Tools/Scripts/webkitpy/tool/steps/postdiffforrevert.py", line 29, in <module>
    from webkitpy.common.net.bugzilla import Attachment
  File "/mnt/c/home/webkit/ga/Tools/Scripts/webkitpy/common/net/bugzilla/__init__.py", line 4, in <module>
    from webkitpy.common.net.bugzilla.bugzilla import Bugzilla
  File "/mnt/c/home/webkit/ga/Tools/Scripts/webkitpy/common/net/bugzilla/bugzilla.py", line 48, in <module>
    from webkitpy.common.net.credentials import Credentials
  File "/mnt/c/home/webkit/ga/Tools/Scripts/webkitpy/common/net/credentials.py", line 44, in <module>
    import keyring
  File "/mnt/c/home/webkit/ga/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 560, in find_module
    cls.install(name)
  File "/mnt/c/home/webkit/ga/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 544, in install
    return all([to_install.install() for to_install in cls.register(package)])
  File "/mnt/c/home/webkit/ga/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 302, in install
    stderr=setup_log,
  File "/usr/lib/python2.7/subprocess.py", line 190, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python', '/tmp/keyring/keyring-7.3.1/setup.py', 'install', '--home=/mnt/c/home/webkit/ga/Tools/Scripts/libraries/autoinstalled/python-2', '--root=/', '--prefix=', '--install-lib=/mnt/c/home/webkit/ga/Tools/Scripts/libraries/autoinstalled/python-2', '--install-scripts=/mnt/c/home/webkit/ga/Tools/Scripts/libraries/autoinstalled/python-2/bin', '--install-data=/mnt/c/home/webkit/ga/Tools/Scripts/libraries/autoinstalled/python-2/data', '--install-headers=/mnt/c/home/webkit/ga/Tools/Scripts/libraries/autoinstalled/python-2/headers']' returned non-zero exit status 1

[Fujii Hironori]

Created attachment 424100 [details]
/tmp/keyring/log.txt on Linux

[Fujii Hironori]

Created attachment 424101 [details]
log.txt on Mac

/var/folders/q5/96sxnd7x3sgf_j0pb5sz3yyw0000gp/T/keyring/log.txt

[Fujii Hironori]

This seems Python 2 specific.

[Sam Sneddon [:gsnedders]]

So the underlying problem is keyring has `setup_requires = setuptools_scm >= 1.15.0` in its setup.py, which leads to setuptools fetching the latest version of setuptools_scm, and setuptools_scm 6 which was released recently drops support for Python 2.

I wonder if bug 223609 upgrading setuptools will aid this (but easy_install is deprecated so still might not have the awareness pip does of Python 2/3 compatibility)?

[Jonathan Bedard]

(In reply to Sam Sneddon [:gsnedders] from comment #4)
> So the underlying problem is keyring has `setup_requires = setuptools_scm >=
> 1.15.0` in its setup.py, which leads to setuptools fetching the latest
> version of setuptools_scm, and setuptools_scm 6 which was released recently
> drops support for Python 2.
> 
> I wonder if bug 223609 upgrading setuptools will aid this (but easy_install
> is deprecated so still might not have the awareness pip does of Python 2/3
> compatibility)?

I'll mess with this a bit today, I see two solutions to this. One is to ensure that we have a newer setuptools that still supports Python 2, the other is to have an older keyring that doesn't require a newer setuptools.

[Radar WebKit Bug Importer]

<rdar://problem/75800504>

[Jonathan Bedard]

Created attachment 424178 [details]
Patch

[Jonathan Bedard]

The attached patch fixes the problem. It's inspired by Sam Sneddon's change in https://bugs.webkit.org/show_bug.cgi?id=223609, but I think we should land this independently because Sam's change needs to be checked against more platforms because it touches more.

Comment on attachment 424178 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=424178&action=review

r=me with comment.

> Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py:235
>          # Make sure that setuptools and wheel are installed, since setup.py relies on it

We may want to update the comment here as well.

[Jonathan Bedard]

Created attachment 424198 [details]
Patch for landing

[EWS]

Committed r274986: <https://commits.webkit.org/r274986>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 424198 [details].