Currently, run-webkit-tests is running several times the same tests when the tests are passed as directories using an absolute path.
Created attachment 238168 [details] Patch
It's not obvious to me what use case this fixes. Could you please provide some examples?
As part of bug 134766 and bug 134767, webkit tests are run outside the LayoutTests folder (inside WebKitBuild folder) to keep the source tree clean.
That sounds like it has quite a few downsides. I think that we should discuss this on webkit-dev prior ot going forward with bugs. - If the tests are transient and don't have a common root, how are we going to report these to flakiness dashboard? - Currently, I can send a trac link to anyone, and a lot of tests will just run in browser. This is very helpful when a test uncovers a bug in a system component, so there is an easy way to reproduce. With transient tests that are not in the tree, that would not work.
(In reply to comment #4) > That sounds like it has quite a few downsides. I think that we should discuss this on webkit-dev prior ot going forward with bugs. Sure, I will do that again. > - If the tests are transient and don't have a common root, how are we going to report these to flakiness dashboard? > > - Currently, I can send a trac link to anyone, and a lot of tests will just run in browser. This is very helpful when a test uncovers a bug in a system component, so there is an easy way to reproduce. With transient tests that are not in the tree, that would not work. The whole W3C test suite would actually be imported within the LayoutTests folder and run as usual webkit tests. The tests with absolute paths would be run offline by people wanting to gather conformance statistics or to bump the imported W3C test suite version and update related test expectations.
I see, that makes more sense to me. Not quite sure why the tree needs to remain clean when doing something like this though - it's easy to revert changes once done.
(In reply to comment #6) > I see, that makes more sense to me. Not quite sure why the tree needs to remain clean when doing something like this though - it's easy to revert changes once done. Running absolute path tests also ensures that all generated files (expected.txt, crash logs, stderr, diff) are located in the same folder as the tests. That makes it a tad easier to compute the conformance results and generate self-contained reports. See http://youennf.github.io/w3c-reports/12092014/w3c_conformance_results.html for instance: the report links each test to its relevant .txt file. We could as well run the tests in a git/svn-ignored LayoutTests sub-folder. The conformance result computation and report generation would probably be somewhat more complex.
Comment on attachment 238168 [details] Patch Clearing flags on attachment: 238168 Committed r173798: <http://trac.webkit.org/changeset/173798>
All reviewed patches have been landed. Closing bug.