Bug 106562 – NRWT still confused about test count with --repeat-each and --iterations

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Jussi Kukkonen (jku)]

A reference call to r-w-t with no "--iterations" or "--repeat-each" results in these calculations:
 Found 9859 tests; running 9015, skipping 844.

And the same call with "--repeat-each=2 --iterations=2"
 Found 9859 tests; running 2253 (4 times each: --repeat-each=2 --iterations=2), skipping 7606.

That is quite wrong, I'll attach a patch.

[Jussi Kukkonen (jku)]

Created attachment 182132 [details]
Patch

[Jussi Kukkonen (jku)]

Adding dpranke and ojan (from bug 95789), maybe either of you wants to review?

[Zan Dobersek]

Comment on attachment 182132 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=182132&action=review

> Tools/Scripts/webkitpy/layout_tests/views/printing.py:101
> +        num_unique_tests = num_to_run

Assuming that this works, you can just use num_to_run wherever num_unique_tests is used.

[Jussi Kukkonen (jku)]

Created attachment 182136 [details]
remove the useless variable as well

[Dirk Pranke]

Comment on attachment 182136 [details]
remove the useless variable as well

Okay, so it appears that this patch changes things so that we print the # of distinct tests we found, the number of those that we'll actually run, the number of unique tests that we'll actually skip, and the --repeat-each and --iterations values, right?

So, we're not printing the aggregate totals, i.e., if we're running 100 tests ten times each, we'll never print 1000.

Did your change break any tests? It sure seems like it should've. If it doesn't, that means that this isn't being tested, and since it's behavior we've probably broken at least twice, we should probably have a test for it (even though this is just a cosmetic thing). Mind adding a test?

[Jussi Kukkonen (jku)]

Created attachment 182833 [details]
Add test for print_found()

[Dirk Pranke]

Comment on attachment 182833 [details]
Add test for print_found()

looks great, thanks!

[WebKit Review Bot]

Comment on attachment 182833 [details]
Add test for print_found()

Clearing flags on attachment: 182833

Committed r139841: <http://trac.webkit.org/changeset/139841>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.

[Dirk Pranke]

Comment on attachment 182833 [details]
Add test for print_found()

View in context: https://bugs.webkit.org/attachment.cgi?id=182833&action=review

> Tools/Scripts/webkitpy/layout_tests/views/printing.py:106
> +        print (found_str)

turns out this got left in accidentally and I missed it in the review. Fixed in http://trac.webkit.org/changeset/139913 .

[Jussi Kukkonen (jku)]

(In reply to comment #10)
> turns out this got left in accidentally and I missed it in the review. Fixed in http://trac.webkit.org/changeset/139913 .

Thanks for handling it.