Bug 210880 – run-webkit-tests should not fail if all tests found to run are skipped.

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Matt Lewis]

run-webkit-tests should not fail if all tests found to run are skipped.
Currently if tests are found to run but all of those tests are skipped, we return a failing value of -1. We should still fail if there are no test to run, but we shouldn't if all tests are skipped.

[Radar WebKit Bug Importer]

<rdar://problem/62213993>

[Jonathan Bedard]

I don't know about this.

Only selecting skipped tests seems like something we want to surface. The exit code really only matter in automation, and if we're skipping all tests, shouldn't we just avoid running that particular set of tests on that particular configuration?

[Matt Lewis]

So I see your point here but after talking with Alexey, this was something we discussed as being a situation that we should't fail on. Perhaps he can provide a bit more behind his thoughts.

[Alexey Proskuryakov]

Skipping and moving tests is a separate decision from what we run in automation. It is possible that we have some tests run one day, and they get moved to a different suite another day. I don't think that we'd want to change buildbot configuration in lockstep.

[Matt Lewis]

Created attachment 399204 [details]
Patch

[Jonathan Bedard]

Comment on attachment 399204 [details]
Patch

r+ing because this behavior is changed correctly, although I disagree that we should be changing this behavior. Since Alexey, Matt and our other bot watchers are for this change, so I'm not going to block it.

[Matt Lewis]

Committed r261629: <https://trac.webkit.org/changeset/261629>