249148 2022-12-12 07:59:46 -0800 import-w3c-tests should work even with broken symlinks in WPT repo 2023-08-02 09:36:56 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Nightly Build Unspecified Unspecified NEW InRadar P5 Normal --- 237724 1 ntim webkit-unassigned ap gsnedders webkit-bug-importer oldest_to_newest 1918492 0 ntim 2022-12-12 07:59:46 -0800 Reverting https://github.com/web-platform-tests/wpt/commit/ef70ce9623f50e9e7a89bb713920588cfb1259cf fixed it, but we should investigate why this caused breakage on the script. Downloading W3C test repositories Fetching https://github.com/web-platform-tests/wpt.git... Checking out revision origin/master Traceback (most recent call last): File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/import-w3c-tests", line 35, in <module> sys.exit(test_importer.main(sys.argv[1:], sys.stdout, sys.stderr)) File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/webkitpy/w3c/test_importer.py", line 98, in main test_importer.do_import() File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/webkitpy/w3c/test_importer.py", line 208, in do_import self.test_downloader().download_tests(self.source_directory, self.test_paths, self.options.use_tip_of_tree) File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/webkitpy/w3c/test_downloader.py", line 228, in download_tests self.copy_tests(destination_directory, test_paths) File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/webkitpy/w3c/test_downloader.py", line 181, in copy_tests self._filesystem.copyfile(source_path, destination_path) File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/webkitpy/common/system/filesystem.py", line 83, in copyfile return shutil.copyfile(source, destination) File "/AppleInternal/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/shutil.py", line 264, in copyfile with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst: FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Data/Code/Safari/OpenSource/WebKitBuild/w3c-tests/web-platform-tests/tools/third_party/funcsigs/docs/index.rst' 1919109 1 gsnedders 2022-12-14 04:10:43 -0800 This is frankly bizarre/worrying, as I don't see why we'd be trying to copy a file that doesn't exist. 1920474 2 webkit-bug-importer 2022-12-19 08:00:17 -0800 <rdar://problem/103516141> 1969280 3 gsnedders 2023-08-02 09:35:38 -0700 As I commented in the upstream PR of the revert (https://github.com/web-platform-tests/wpt/pull/37442): > [ef70ce9](https://github.com/web-platform-tests/wpt/commit/ef70ce9623f50e9e7a89bb713920588cfb1259cf) left `tools/third_party/funcsigs/docs/index.rst` as a symlink to the (non-existent, removed) `tools/third_party/funcsigs/docs/README.rst`, which I believe is the origin of all the problems here. Given WPT _shouldn't_ have any broken symlinks, this is very low priority, and we're better off fixing this in the near-term by just reverting any change to WPT that introduces and broken symlinks. The importer bug here would be easy to fix with the Python 3.3+ follow_symlinks argument on `shutil.copyfile(src, dst, *, follow_symlinks=True)`, as we just need to pass `follow_symlinks=False` in copy_tests in test_downloader.py (and pass it though webkitpy.common.system.filesystem), but as long as we support Python 2 this is more effort than is probably justified. For the record, https://github.com/web-platform-tests/wpt/pull/41292 re-lands the funcsigs drop _with_ the symlink actually removed this time.