Bug 92527 – nrwt: clean up handling of 'expected' stats in the non-default cases

• Component: New Bugs
• Status: RESOLVED FIXED

[Dirk Pranke]

nrwt: clean up handling of 'expected' stats and interaction with --run-part and --run-chunk

[Dirk Pranke]

Created attachment 155017 [details]
Patch

[Dirk Pranke]

the interplay of --iterations, --repeat-each, -run-part, and -run-chunk with the way we logged how many tests to run and what we expected the results to be was confusing and badly layered.

This patch attempts to clean that up, and was necessary in order to split the finding, chunking, and sharding of tests out from the running of tests without having all sorts of weird callbacks.

Apart from logging slightly different numbers, there should be no functional changes in this patch; in particular; the contents of results.json should be unchanged.

[Tony Chang]

Comment on attachment 155017 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=155017&action=review

> Tools/Scripts/webkitpy/layout_tests/models/test_expectations.py:887
> +    def add_skipped_tests(self, tests_to_skip, wontfix=True):

I don't see either of the callers passing in False for wontfix.  Can we drop this param?

[Dirk Pranke]

(In reply to comment #3)
> (From update of attachment 155017 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=155017&action=review
> 
> > Tools/Scripts/webkitpy/layout_tests/models/test_expectations.py:887
> > +    def add_skipped_tests(self, tests_to_skip, wontfix=True):
> 
> I don't see either of the callers passing in False for wontfix.  Can we drop this param?

Good catch. It's not clear when tests skipped via the Skipped file, the command line, or because we're only running parts of the test should be WONTFIX or not, but I'm not sure it matters that much, either.

[Dirk Pranke]

Created attachment 155400 [details]
remove wontfix from add_skipped_tests, expectation_for_skipped_tests

[Dirk Pranke]

Committed r124116: <http://trac.webkit.org/changeset/124116>