Bug 249148 – import-w3c-tests should work even with broken symlinks in WPT repo

• Status: NEW
• Component: Tools / Tests

[Tim Nguyen (:ntim)]

Reverting https://github.com/web-platform-tests/wpt/commit/ef70ce9623f50e9e7a89bb713920588cfb1259cf fixed it, but we should investigate why this caused breakage on the script.

Downloading W3C test repositories
Fetching https://github.com/web-platform-tests/wpt.git...
Checking out revision origin/master
Traceback (most recent call last):
  File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/import-w3c-tests", line 35, in <module>
    sys.exit(test_importer.main(sys.argv[1:], sys.stdout, sys.stderr))
  File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/webkitpy/w3c/test_importer.py", line 98, in main
    test_importer.do_import()
  File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/webkitpy/w3c/test_importer.py", line 208, in do_import
    self.test_downloader().download_tests(self.source_directory, self.test_paths, self.options.use_tip_of_tree)
  File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/webkitpy/w3c/test_downloader.py", line 228, in download_tests
    self.copy_tests(destination_directory, test_paths)
  File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/webkitpy/w3c/test_downloader.py", line 181, in copy_tests
    self._filesystem.copyfile(source_path, destination_path)
  File "/Volumes/Data/Code/Safari/OpenSource/Tools/Scripts/webkitpy/common/system/filesystem.py", line 83, in copyfile
    return shutil.copyfile(source, destination)
  File "/AppleInternal/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/shutil.py", line 264, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Data/Code/Safari/OpenSource/WebKitBuild/w3c-tests/web-platform-tests/tools/third_party/funcsigs/docs/index.rst'

[Sam Sneddon [:gsnedders]]

This is frankly bizarre/worrying, as I don't see why we'd be trying to copy a file that doesn't exist.

[Radar WebKit Bug Importer]

<rdar://problem/103516141>

[Sam Sneddon [:gsnedders]]

As I commented in the upstream PR of the revert (https://github.com/web-platform-tests/wpt/pull/37442):

> [ef70ce9](https://github.com/web-platform-tests/wpt/commit/ef70ce9623f50e9e7a89bb713920588cfb1259cf) left `tools/third_party/funcsigs/docs/index.rst` as a symlink to the (non-existent, removed) `tools/third_party/funcsigs/docs/README.rst`, which I believe is the origin of all the problems here.

Given WPT _shouldn't_ have any broken symlinks, this is very low priority, and we're better off fixing this in the near-term by just reverting any change to WPT that introduces and broken symlinks.

The importer bug here would be easy to fix with the Python 3.3+ follow_symlinks argument on `shutil.copyfile(src, dst, *, follow_symlinks=True)`, as we just need to pass `follow_symlinks=False` in copy_tests in test_downloader.py (and pass it though webkitpy.common.system.filesystem), but as long as we support Python 2 this is more effort than is probably justified.

For the record, https://github.com/web-platform-tests/wpt/pull/41292 re-lands the funcsigs drop _with_ the symlink actually removed this time.