Bug 226097 – Run layout tests in stress mode for 10 iterations on regular EWS layout-test queues

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Aakash Jain]

Stress mode for layout-test was added as a stand-alone queue in https://lists.webkit.org/pipermail/webkit-dev/2021-April/031773.html

Running for 100 iterations sometimes takes long time for certain patches. However we should run the tests being added/modified for limited iterations (e.g.: 10) on regular EWS layout-test queues. This would provide following benefit: 

1) some coverage for stress test on multiple platforms
2) early indication of issues (since running 10 iterations of layout-tests being added/modified is usually quick), before running the complete layout-test suite.

[Aakash Jain]

Created attachment 429317 [details]
Patch

[Aakash Jain]

Sample runs:

https://ews-build.webkit-uat.org/#/builders/34/builds/31233 (on patch which modify layout-test)
https://ews-build.webkit-uat.org/#/builders/34/builds/31254 (on patch which doesn't modify layout-test)

[Jonathan Bedard]

Comment on attachment 429317 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=429317&action=review

> Tools/CISupport/ews-build/steps_unittest.py:1947
> +                                 '--iterations', 100, 'test1', 'test2'],

I'm a bit unclear why anything as changed here.

[Jonathan Bedard]

Comment on attachment 429317 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=429317&action=review

>> Tools/CISupport/ews-build/steps_unittest.py:1947
>> +                                 '--iterations', 100, 'test1', 'test2'],
> 
> I'm a bit unclear why anything as changed here.

oh! Because you added the "modified_tests" property, got it.

[Aakash Jain]

Yeah. This should have been there in the first place. But adding the 'doStepIf' in this patch (based on modified_tests) makes it required now.

[EWS]

Committed r277884 (238021@main): <https://commits.webkit.org/238021@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 429317 [details].

[Radar WebKit Bug Importer]

<rdar://problem/78327180>

[Aakash Jain]

Restarted EWS to pick up this change.

[Aakash Jain]

This change seems to be working fine. e.g.:

Patch which doesn't modify layout-test: https://ews-build.webkit.org/#/builders/51/builds/14126

Patch which add/modify layout-tests and passes: https://ews-build.webkit.org/#/builders/51/builds/14125

Patch which add/modify layout-tests and introduce failure: https://ews-build.webkit.org/#/builders/51/builds/14098