Bug 271186 – REGRESSION(276129@main): `run-webkit-tests imported/w3c/web-platform-tests/css/*/*animation*` doesn't find tests to run

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Sam Sneddon [:gsnedders]]

rdar://124918090

I swear I was able to run tests using multiple `*` in the path like so:

```
rwt --debug imported/w3c/web-platform-tests/css/*/*animation*
```

Now this doesn’t work:

```
OpenSource > rwt --debug imported/w3c/web-platform-tests/css/*/*animation*
Using port 'mac-sonoma-wk2'
Test configuration: <sonoma, arm64, debug>
Placing test results in /Users/antoine/Builds/Debug/layout-test-results
Using Debug build
Pixel tests disabled
Regular timeout: 30000, slow test timeout: 150000
Command line: /Users/antoine/Builds/Debug/WebKitTestRunner -

Collecting tests ... 
```

Adding a `LayoutTests/` prefix makes it work though:

```
rwt --debug LayoutTests/imported/w3c/web-platform-tests/css/*/*animation*
```

This command finds 457 tests to run.

This regressed with https://commits.webkit.org/276129@main, the fix for rdar://72894403 (LayoutTestFinder should be in charge of finding expectations for each test (220421)).

---

What shell are you using? Because I’d expect it to fail with the shell being unable to expand the globs, without it even getting to run-webkit-tests:

gsnedders@gsnedders-marsha OpenSource % ./Tools/Scripts/run-webkit-tests --debug --print-expectations imported/w3c/web-platform-tests/css/*/*animation*
zsh: no matches found: imported/w3c/web-platform-tests/css/*/*animation*

With it quoted, I get:

% ./Tools/Scripts/run-webkit-tests --debug --print-expectations 'imported/w3c/web-platform-tests/css/*/*animation*'
Collecting tests ...%

Which *is* more suspicious.

Fixing the swallowing of some exception in https://bugs.webkit.org/show_bug.cgi?id=271180, because after we fix that we see:

Traceback (most recent call last):
  File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 94, in main
    run_details = run(port, options, args, stderr)
  File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 556, in run
    run_details = manager.run(args)
  File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 328, in run
    tests_to_run_by_device, aggregate_tests_to_skip = self._collect_tests(args, device_type_list)
  File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 123, in _collect_tests
    paths, tests = self._finder.find_tests(self._options, paths, device_type=device_type)
  File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder_legacy.py", line 67, in find_tests
    tests = self.find_tests_by_path(paths, device_type=device_type, with_expectations=with_expectations)
  File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder_legacy.py", line 78, in find_tests_by_path
    return list(finder.get_tests(paths))
  File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py", line 210, in get_tests
    for item in self._process_directory(
  File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py", line 319, in _process_directory
    it = self.fs.scandir(current_layout_tests_path)
  File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/common/system/filesystem.py", line 195, in scandir
    return os.scandir(path)
NotADirectoryError: [Errno 20] Not a directory: '/Volumes/gsnedders/projects/Safari/OpenSource/LayoutTests/imported/w3c/web-platform-tests/css/w3c-import.log'

…which is an accurate observation. Pretty sure we just need to check whether the glob matches a directory (v. a file) in one place.

[Radar WebKit Bug Importer]

<rdar://problem/124970843>

[Sam Sneddon [:gsnedders]]

rdar://124918090

[Sam Sneddon [:gsnedders]]

Pull request: https://github.com/WebKit/WebKit/pull/26077

[EWS]

Committed 276306@main (399ae11e2665): <https://commits.webkit.org/276306@main>

Reviewed commits have been landed. Closing PR #26077 and removing active labels.