285315 2025-01-02 13:10:05 -0800 'git-webkit setup' fails with "ValueError: No archives for MarkupSafe-2.1.5 found" 2025-01-10 08:41:44 -0800 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED InRadar P2 Normal --- 281877 1 mcatanzaro mcatanzaro gsnedders jbedard mcatanzaro webkit-bug-importer oldest_to_newest 2084546 0 mcatanzaro 2025-01-02 13:10:05 -0800 It seems 'git-webkit setup' is broken: $ git-webkit setup Traceback (most recent call last): File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/git-webkit", line 63, in <module> sys.exit(program.main( ~~~~~~~~~~~~^ identifier_template=is_webkit_filter('Canonical link: https://commits.webkit.org/{}'), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... classifier=is_webkit_filter(classifier()), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ )) ^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/program/__init__.py", line 184, in main return parsed.main( ~~~~~~~~~~~^ args=parsed, ^^^^^^^^^^^^ ...<5 lines>... canonical_svn=canonical_svn, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/program/setup.py", line 621, in main result = cls.git(args, repository, **kwargs) File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/program/setup.py", line 432, in git result += InstallHooks.main(args, repository, hooks=hooks, **kwargs) ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/program/install_hooks.py", line 230, in main from jinja2 import Template File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1322, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 1262, in _find_spec File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 697, in find_spec loader = cls.find_module(fullname, path=path) File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 711, in find_module cls.install(name) ~~~~~~~~~~~^^^^^^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 684, in install return all([to_install.install() for to_install in packages]) ~~~~~~~~~~~~~~~~~~^^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 308, in install AutoInstall.install(dependency) ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 684, in install return all([to_install.install() for to_install in packages]) ~~~~~~~~~~~~~~~~~~^^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 322, in install raise ValueError('No archives for {}-{} found'.format(self.pypi_name, self.version)) ValueError: No archives for MarkupSafe-2.1.5 found I'm using python 3.13.0. No clue why it's not able to find MarkupSafe 2.1.5, which seems to be normally available from PyPi. 2085865 1 gsnedders 2025-01-08 11:45:28 -0800 I wish we gave better errors here. https://pypi.org/project/MarkupSafe/2.1.5/#files shows there's no wheel for Python 3.13, and we seem to be forcing binary installs: Tools/Scripts/libraries/webkitflaskpy/webkitflaskpy/__init__.py 57:AutoInstall.register(Package('markupsafe', Version(2, 1, 5), pypi_name='MarkupSafe', wheel=True)) Tools/Scripts/libraries/webkitscmpy/webkitscmpy/__init__.py 56:AutoInstall.register(Package('markupsafe', Version(2, 1, 5), pypi_name='MarkupSafe', wheel=True)) Probably just need to bump these for 3.13, but that implies a major version upgrade — try and run the tests, I guess? 2085899 2 mcatanzaro 2025-01-08 15:04:27 -0800 I tried running test-webkitpy but it fails: Traceback (most recent call last): File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/webkitpy/test/main.py", line 392, in <module> sys.exit(main()) ~~~~^^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/webkitpy/test/main.py", line 115, in main return not tester.run() ~~~~~~~~~~^^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/webkitpy/test/main.py", line 193, in run return self._run_tests(names) ~~~~~~~~~~~~~~~^^^^^^^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/webkitpy/test/main.py", line 207, in _run_tests AutoInstall.install_everything() ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 696, in install_everything package.install() ~~~~~~~~~~~~~~~^^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 322, in install raise ValueError('No archives for {}-{} found'.format(self.pypi_name, self.version)) ValueError: No archives for hiredis-3 found which is fixed by upgrading to hiredis-3.1.0, and also specifies wheel=true, so presumably that's the same problem. Next error: Traceback (most recent call last): File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/webkitpy/test/main.py", line 392, in <module> sys.exit(main()) ~~~~^^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/webkitpy/test/main.py", line 115, in main return not tester.run() ~~~~~~~~~~^^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/webkitpy/test/main.py", line 193, in run return self._run_tests(names) ~~~~~~~~~~~~~~~^^^^^^^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/webkitpy/test/main.py", line 207, in _run_tests AutoInstall.install_everything() ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 696, in install_everything package.install() ~~~~~~~~~~~~~~~^^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 322, in install raise ValueError('No archives for {}-{} found'.format(self.pypi_name, self.version)) ValueError: No archives for lupa-2.2 found But lupa is already at the latest version and there is just no wheel available. If I remove wheel=true, then our autoinstaller compiles it successfully instead of using the prebuilt wheel, so I guess we should do what works. The webkitpy test suite fails, though. Several tests raise the following error: 602/2387] webkitpy.layout_tests.controllers.layout_test_finder_legacy_unittest.LayoutTestFinderTests.test_tempdir_relpath failed: Traceback (most recent call last): File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder_legacy_unittest.py", line 1585, in test_tempdir_relpath with fs.mkdtemp() as path: ~~~~~~~~~~^^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/webkitpy/common/system/filesystem.py", line 213, in __exit__ self._filesystem.rmtree(self._directory_path) ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^ File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/webkitpy/common/system/filesystem.py", line 316, in rmtree shutil.rmtree(path, ignore_errors=True) ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.13/shutil.py", line 763, in rmtree _rmtree_safe_fd(stack, onexc) ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^ File "/usr/lib64/python3.13/shutil.py", line 663, in _rmtree_safe_fd assert func is os.lstat ^^^^^^^^^^^^^^^^ This looks like a bug in the python standard library. Then the test suite hangs after executing 2353 out of 2387 tests, but by this point I wasn't really expecting it to work, so OK. :) My changes thus far: diff --git a/Tools/Scripts/libraries/webkitflaskpy/webkitflaskpy/__init__.py b/Tools/Scripts/libraries/webkitflaskpy/webkitflaskpy/__init__.py index f172250e7c3f..60d3ad9df042 100644 --- a/Tools/Scripts/libraries/webkitflaskpy/webkitflaskpy/__init__.py +++ b/Tools/Scripts/libraries/webkitflaskpy/webkitflaskpy/__init__.py @@ -52,11 +52,11 @@ version = Version(0, 7, 1) AutoInstall.register(Package('blinker', Version(1, 8, 2))) AutoInstall.register(Package('click', Version(8, 1, 7))) AutoInstall.register(Package('flask', Version(2, 3, 3))) -AutoInstall.register(Package('hiredis', Version(3, 0, 0), wheel=True)) +AutoInstall.register(Package('hiredis', Version(3, 1, 0), wheel=True)) AutoInstall.register(Package('itsdangerous', Version(1, 1, 0))) -AutoInstall.register(Package('markupsafe', Version(2, 1, 5), pypi_name='MarkupSafe', wheel=True)) +AutoInstall.register(Package('markupsafe', Version(3, 0, 2), pypi_name='MarkupSafe', wheel=True)) AutoInstall.register(Package('jinja2', Version(3, 1, 4), implicit_deps=['markupsafe'])) -AutoInstall.register(Package('lupa', Version(2, 2), wheel=True)) +AutoInstall.register(Package('lupa', Version(2, 2))) AutoInstall.register(Package('redis', Version(3, 5, 3))) AutoInstall.register(Package('sortedcontainers', Version(2, 4, 0))) AutoInstall.register(Package('werkzeug', Version(3, 0, 3))) diff --git a/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/__init__.py b/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/__init__.py index fd1dc5abed72..2bb03a2f05f3 100644 --- a/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/__init__.py +++ b/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/__init__.py @@ -53,7 +53,7 @@ except ImportError: version = Version(7, 0, 0) AutoInstall.register(Package('fasteners', Version(0, 15, 0))) -AutoInstall.register(Package('markupsafe', Version(2, 1, 5), pypi_name='MarkupSafe', wheel=True)) +AutoInstall.register(Package('markupsafe', Version(3, 0, 2), pypi_name='MarkupSafe', wheel=True)) AutoInstall.register(Package('jinja2', Version(3, 1, 4), implicit_deps=['markupsafe'])) AutoInstall.register(Package('monotonic', Version(1, 5))) AutoInstall.register(Package('xmltodict', Version(0, 11, 0))) 2086144 3 gsnedders 2025-01-09 10:53:25 -0800 My suggestion would be to exclude reporelaypy, resultsdbpy, and webkitflaskpy from test-webkitpy (in Tools/Scripts/webkitpy/test/main.py); they're all server-side code and not that relevant in the short-term. I thought we'd excluded them with some earlier release (3.12?), but I can't find such a thing in the history. The failure in rmtree might be fixed by updating pyfakefs, because I think this will have been fixed as a result of https://github.com/pytest-dev/pyfakefs/commit/816d1142e1fb67bbc7526cd8f5f061e00ae8b5b4 2086174 4 webkit-bug-importer 2025-01-09 13:11:16 -0800 <rdar://problem/142637619> 2086175 5 mcatanzaro 2025-01-09 13:17:58 -0800 Pull request: https://github.com/WebKit/WebKit/pull/38804 2086363 6 ews-feeder 2025-01-10 08:41:42 -0800 Committed 288712@main (bef0ec308c65): <https://commits.webkit.org/288712@main> Reviewed commits have been landed. Closing PR #38804 and removing active labels.