271186 2024-03-18 15:03:06 -0700 REGRESSION(276129@main): `run-webkit-tests imported/w3c/web-platform-tests/css/*/*animation*` doesn't find tests to run 2024-03-18 15:29:04 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED InRadar P2 Normal --- 1 gsnedders gsnedders webkit-bug-importer oldest_to_newest 2021907 0 gsnedders 2024-03-18 15:03:06 -0700 rdar://124918090 I swear I was able to run tests using multiple `*` in the path like so: ``` rwt --debug imported/w3c/web-platform-tests/css/*/*animation* ``` Now this doesn’t work: ``` OpenSource > rwt --debug imported/w3c/web-platform-tests/css/*/*animation* Using port 'mac-sonoma-wk2' Test configuration: <sonoma, arm64, debug> Placing test results in /Users/antoine/Builds/Debug/layout-test-results Using Debug build Pixel tests disabled Regular timeout: 30000, slow test timeout: 150000 Command line: /Users/antoine/Builds/Debug/WebKitTestRunner - Collecting tests ... ``` Adding a `LayoutTests/` prefix makes it work though: ``` rwt --debug LayoutTests/imported/w3c/web-platform-tests/css/*/*animation* ``` This command finds 457 tests to run. This regressed with https://commits.webkit.org/276129@main, the fix for rdar://72894403 (LayoutTestFinder should be in charge of finding expectations for each test (220421)). --- What shell are you using? Because I’d expect it to fail with the shell being unable to expand the globs, without it even getting to run-webkit-tests: gsnedders@gsnedders-marsha OpenSource % ./Tools/Scripts/run-webkit-tests --debug --print-expectations imported/w3c/web-platform-tests/css/*/*animation* zsh: no matches found: imported/w3c/web-platform-tests/css/*/*animation* With it quoted, I get: % ./Tools/Scripts/run-webkit-tests --debug --print-expectations 'imported/w3c/web-platform-tests/css/*/*animation*' Collecting tests ...% Which *is* more suspicious. Fixing the swallowing of some exception in https://bugs.webkit.org/show_bug.cgi?id=271180, because after we fix that we see: Traceback (most recent call last): File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 94, in main run_details = run(port, options, args, stderr) File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 556, in run run_details = manager.run(args) File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 328, in run tests_to_run_by_device, aggregate_tests_to_skip = self._collect_tests(args, device_type_list) File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 123, in _collect_tests paths, tests = self._finder.find_tests(self._options, paths, device_type=device_type) File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder_legacy.py", line 67, in find_tests tests = self.find_tests_by_path(paths, device_type=device_type, with_expectations=with_expectations) File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder_legacy.py", line 78, in find_tests_by_path return list(finder.get_tests(paths)) File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py", line 210, in get_tests for item in self._process_directory( File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py", line 319, in _process_directory it = self.fs.scandir(current_layout_tests_path) File "/Volumes/gsnedders/projects/Safari/OpenSource/Tools/Scripts/webkitpy/common/system/filesystem.py", line 195, in scandir return os.scandir(path) NotADirectoryError: [Errno 20] Not a directory: '/Volumes/gsnedders/projects/Safari/OpenSource/LayoutTests/imported/w3c/web-platform-tests/css/w3c-import.log' …which is an accurate observation. Pretty sure we just need to check whether the glob matches a directory (v. a file) in one place. 2021908 1 webkit-bug-importer 2024-03-18 15:03:15 -0700 <rdar://problem/124970843> 2021910 2 gsnedders 2024-03-18 15:05:47 -0700 rdar://124918090 2021911 3 gsnedders 2024-03-18 15:06:22 -0700 Pull request: https://github.com/WebKit/WebKit/pull/26077 2021922 4 ews-feeder 2024-03-18 15:29:02 -0700 Committed 276306@main (399ae11e2665): <https://commits.webkit.org/276306@main> Reviewed commits have been landed. Closing PR #26077 and removing active labels.