Orphaned files? This detects "-expected.*" files which doesn't seem to have an associated test file.
Created attachment 84252 [details] [SCRIPT] Find Expected Results Without Test Script I used to find the issues.
Created attachment 84253 [details] [OUTPUT] Missing Tests Summary: 426 expected results missing tests. 500 unpaired pngs/checksums. Should all "-expected.png" files have an "-expected.checksum"? I print those out as warnings. Also, if there is a test in "platform/x/.../foo.html" I assume that test can never have expected results outside of platform x. If that isn't the case, like a chain of platform dependencies, then this could be slightly wrong.
(In reply to comment #2) > Should all "-expected.png" files have an "-expected.checksum"? I print those > out as warnings. Yes, that's the fast path for determining if -actual.png and -expected.png results match. > Also, if there is a test in "platform/x/.../foo.html" I assume that test can never > have expected results outside of platform x. If that isn't the case, like a > chain of platform dependencies, then this could be slightly wrong. That is correct. Although I suppose platform/mac could have different results in platform/mac-leopard and platform/mac-snowleopard, although I'm not sure whether [old-]run-webkit-tests actually handles this or not.
(In reply to comment #1) > Created an attachment (id=84252) [details] > [SCRIPT] Find Expected Results Without Test > > Script I used to find the issues. You should also put this up for review!
I noticed at least one issue with my script. There should be a list of fallback directories to search in. # For expected result: LayoutTests/platform/chromium-gpu-win/foo/bar/x-expected.txt we check: # # LayoutTests/platform/chromium-gpu-win/foo/bar/x.* # LayoutTests/platform/chromium-gpu/foo/bar/x.* # LayoutTests/platform/chromium/foo/bar/x.* # LayoutTests/foo/bar/x.* # # If no test file is found its an orphan! Updated this and I now only see: 393 expected results missing tests. 500 unpaired pngs/checksums. I'll do a little more verification before writing a patch to remove the excess files.
Heh, doing a quick verification I found 1 weird expected result. Turns out my script was working correctly: LayoutTests/platform/mac-leopard/platform/mac/svg/batik/text/smallFonts-expected.txt Should be nixed because no test exists at: LayoutTests/platform/mac/svg/batik/text/smallFonts.* So I think I'll write up a patch to remove these.
Turns out the fallback behaviors, at least in run-webkit-tests, are quite weird. They are defined in Tools/Scripts/webkitpy/layout_tests/ports/*. So, a perfect script would need to mimic deduplicate-tests, and follow the idiosyncrasies in the fallback behavior of some ports.