Bug 226097 - Run layout tests in stress mode for 10 iterations on regular EWS layout-test queues
Summary: Run layout tests in stress mode for 10 iterations on regular EWS layout-test ...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Aakash Jain
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-05-21 11:49 PDT by Aakash Jain
Modified: 2021-05-26 08:27 PDT (History)
6 users (show)

See Also:


Attachments
Patch (14.13 KB, patch)
2021-05-21 11:51 PDT, Aakash Jain
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Aakash Jain 2021-05-21 11:49:58 PDT
Stress mode for layout-test was added as a stand-alone queue in https://lists.webkit.org/pipermail/webkit-dev/2021-April/031773.html

Running for 100 iterations sometimes takes long time for certain patches. However we should run the tests being added/modified for limited iterations (e.g.: 10) on regular EWS layout-test queues. This would provide following benefit: 

1) some coverage for stress test on multiple platforms
2) early indication of issues (since running 10 iterations of layout-tests being added/modified is usually quick), before running the complete layout-test suite.
Comment 1 Aakash Jain 2021-05-21 11:51:29 PDT
Created attachment 429317 [details]
Patch
Comment 2 Aakash Jain 2021-05-21 12:45:01 PDT
Sample runs:

https://ews-build.webkit-uat.org/#/builders/34/builds/31233 (on patch which modify layout-test)
https://ews-build.webkit-uat.org/#/builders/34/builds/31254 (on patch which doesn't modify layout-test)
Comment 3 Jonathan Bedard 2021-05-21 13:24:28 PDT
Comment on attachment 429317 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=429317&action=review

> Tools/CISupport/ews-build/steps_unittest.py:1947
> +                                 '--iterations', 100, 'test1', 'test2'],

I'm a bit unclear why anything as changed here.
Comment 4 Jonathan Bedard 2021-05-21 13:27:06 PDT
Comment on attachment 429317 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=429317&action=review

>> Tools/CISupport/ews-build/steps_unittest.py:1947
>> +                                 '--iterations', 100, 'test1', 'test2'],
> 
> I'm a bit unclear why anything as changed here.

oh! Because you added the "modified_tests" property, got it.
Comment 5 Aakash Jain 2021-05-21 13:28:12 PDT
Yeah. This should have been there in the first place. But adding the 'doStepIf' in this patch (based on modified_tests) makes it required now.
Comment 6 EWS 2021-05-21 13:58:11 PDT
Committed r277884 (238021@main): <https://commits.webkit.org/238021@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 429317 [details].
Comment 7 Radar WebKit Bug Importer 2021-05-21 13:59:19 PDT
<rdar://problem/78327180>
Comment 8 Aakash Jain 2021-05-22 04:32:11 PDT
Restarted EWS to pick up this change.
Comment 9 Aakash Jain 2021-05-25 10:39:47 PDT
This change seems to be working fine. e.g.:

Patch which doesn't modify layout-test: https://ews-build.webkit.org/#/builders/51/builds/14126

Patch which add/modify layout-tests and passes: https://ews-build.webkit.org/#/builders/51/builds/14125

Patch which add/modify layout-tests and introduce failure: https://ews-build.webkit.org/#/builders/51/builds/14098