Take the Webkit trunk r144906 as an example. We have efl/TestExpectations:webkit.org/b/84333 fast/files/workers/worker-read-blob-async.html [ Failure ] and platform/wk2/TestExpectations:fast/files/workers/worker-read-blob-async.html We resolve the tests as "efl-wk2 -> wk2 -> efl -> generic" which means this test is being skipped on our WK2 bots, but is showing up on the flakiness dashboard as not failing on the past 100 runs.
I don't think I understand what you're asking for ... Are you asking for a change to lint-test-files, or the flakiness dashboard or both? As far as lint-test-files goes, almost by design files later in the cascade are expected to have entries that override files earlier in the cascade (the original design for the cascade was callled "overrides"). So I would fully expect there to be entries overriding earlier files, and having lint-test-files warn about them would create a lot of noise. Now, it is true that for non-chromium ports (meaning, ports that don't have keywords to limit the scope of a result), in order to scope an expectation to a particular port you *have* to use the cascade, and so there are cases where the expectations aren't overrides (or, at least, they're overriding "PASS"). But I don't know how you'd tell the difference between these two cases and avoid the noise. Perhaps this makes more sense to warn for individual test cases (e.g. with webkit-patch print-expectations to have a flag or something), I dunno. As far as the flakiness dashboard goes, are you saying that the showUnexpectedPasses page shouldn't be listing tests that are Skipped by default? I think that's probably a bug; it looks like the skipped=True flag is broken.