Bug 249148 - import-w3c-tests should work even with broken symlinks in WPT repo
Summary: import-w3c-tests should work even with broken symlinks in WPT repo
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P5 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on: 237724
Blocks:
  Show dependency treegraph
 
Reported: 2022-12-12 07:59 PST by Tim Nguyen (:ntim)
Modified: 2023-08-02 09:36 PDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Tim Nguyen (:ntim) 2022-12-12 07:59:46 PST
Reverting https://github.com/web-platform-tests/wpt/commit/ef70ce9623f50e9e7a89bb713920588cfb1259cf fixed it, but we should investigate why this caused breakage on the script.

Downloading W3C test repositories
Fetching https://github.com/web-platform-tests/wpt.git...
Checking out revision origin/master
Traceback (most recent call last):
  File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/import-w3c-tests", line 35, in <module>
    sys.exit(test_importer.main(sys.argv[1:], sys.stdout, sys.stderr))
  File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/webkitpy/w3c/test_importer.py", line 98, in main
    test_importer.do_import()
  File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/webkitpy/w3c/test_importer.py", line 208, in do_import
    self.test_downloader().download_tests(self.source_directory, self.test_paths, self.options.use_tip_of_tree)
  File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/webkitpy/w3c/test_downloader.py", line 228, in download_tests
    self.copy_tests(destination_directory, test_paths)
  File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/webkitpy/w3c/test_downloader.py", line 181, in copy_tests
    self._filesystem.copyfile(source_path, destination_path)
  File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/webkitpy/common/system/filesystem.py", line 83, in copyfile
    return shutil.copyfile(source, destination)
  File "/AppleInternal/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/shutil.py", line 264, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Data/Code/Safari/OpenSource/WebKitBuild/w3c-tests/web-platform-tests/tools/third_party/funcsigs/docs/index.rst'
Comment 1 Sam Sneddon [:gsnedders] 2022-12-14 04:10:43 PST
This is frankly bizarre/worrying, as I don't see why we'd be trying to copy a file that doesn't exist.
Comment 2 Radar WebKit Bug Importer 2022-12-19 08:00:17 PST
<rdar://problem/103516141>
Comment 3 Sam Sneddon [:gsnedders] 2023-08-02 09:35:38 PDT
As I commented in the upstream PR of the revert (https://github.com/web-platform-tests/wpt/pull/37442):

> [ef70ce9](https://github.com/web-platform-tests/wpt/commit/ef70ce9623f50e9e7a89bb713920588cfb1259cf) left `tools/third_party/funcsigs/docs/index.rst` as a symlink to the (non-existent, removed) `tools/third_party/funcsigs/docs/README.rst`, which I believe is the origin of all the problems here.

Given WPT _shouldn't_ have any broken symlinks, this is very low priority, and we're better off fixing this in the near-term by just reverting any change to WPT that introduces and broken symlinks.

The importer bug here would be easy to fix with the Python 3.3+ follow_symlinks argument on `shutil.copyfile(src, dst, *, follow_symlinks=True)`, as we just need to pass `follow_symlinks=False` in copy_tests in test_downloader.py (and pass it though webkitpy.common.system.filesystem), but as long as we support Python 2 this is more effort than is probably justified.

For the record, https://github.com/web-platform-tests/wpt/pull/41292 re-lands the funcsigs drop _with_ the symlink actually removed this time.