Bug 88947 – nrwt: implement the actual cascade of TestExpectations

• Component: New Bugs
• Status: RESOLVED FIXED

[Dirk Pranke]

nrwt: implement the actual cascade of TestExpectations

[Dirk Pranke]

Created attachment 147218 [details]
Patch

[Ojan Vafai]

Comment on attachment 147218 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=147218&action=review

> Tools/ChangeLog:15
> +        There is an actual semantic change in this patch, in that
> +        setting an expectation on a directory in one file will override
> +        the expectations on any individual tests set in prior files. The
> +        test_overrides__directory() unit test verifies this.

I really think we should simplify the logic of test_expectations entirely (in a separate patch obviously) to last one wins. Even within the same expectations file. The "more specific wins" is more confusing than the benefit it brings.

> Tools/Scripts/webkitpy/layout_tests/models/test_expectations.py:597
> +        if prev_expectation_line.filename != expectation_line.filename:

Clever!

[Dirk Pranke]

(In reply to comment #2)
> I really think we should simplify the logic of test_expectations entirely (in a separate patch obviously) to last one wins. Even within the same expectations file. The "more specific wins" is more confusing than the benefit it brings.

Yeah, as we've discussed before, it's hard to say what the right answer is here. Once we get chromium moved to a world where long-standing expectations are checked in and the file becomes much shorter, you're probably right. In the meantime, I think allowing duplicates in the file might just make it less maintainable.

Hopefully the checked-in-failures world is not far away and we'll be able to test this theory.

[Dirk Pranke]

Committed r120243: <http://trac.webkit.org/changeset/120243>