Bug 197563 – run-api-tests: Add --iterations and --repeat-each

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Shawn Roberts]

Would be nice, if like with run-webkit-tests we could pass --iterations to test for flaky API behavior.

[Radar WebKit Bug Importer]

<rdar://problem/50446491>

[Jonathan Bedard]

Created attachment 368940 [details]
Patch

[Jonathan Bedard]

I'm actually surprised we don't already do this.

In any case, it's an easy enough change.

[EWS Watchlist]

Comment on attachment 368940 [details]
Patch

Attachment 368940 [details] did not pass win-ews (win):
Output: https://webkit-queues.webkit.org/results/12090443

New failing tests:
fast/shadow-dom/svg-text-path-href-change-in-shadow-tree.html

[EWS Watchlist]

Created attachment 368972 [details]
Archive of layout-test-results from ews210 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews210  Port: win-future  Platform: CYGWIN_NT-10.0-17763-3.0.5-338.x86_64-x86_64-64bit

[Aakash Jain]

Comment on attachment 368940 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=368940&action=review

> Tools/Scripts/webkitpy/api_tests/manager.py:185
> +        test_names = [test for test in test_names for _ in xrange(self._options.repeat_each)]

It feels little weird to duplicate the list of test_names. Maybe a cleaner approach would be to run a for loop, and also log a message with iteration number, so that it's easier for someone to find out when a iteration is complete.

> Tools/Scripts/webkitpy/api_tests/manager.py:186
> +        test_names *= self._options.iterations

Maybe worth adding a log indicating the number of iterations.

[Jonathan Bedard]

Created attachment 369029 [details]
Patch for landing

[WebKit Commit Bot]

Comment on attachment 369029 [details]
Patch for landing

Clearing flags on attachment: 369029

Committed r244942: <https://trac.webkit.org/changeset/244942>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.