Bug 260020 - [ews] mac-wk2 build marked as green even though it found failure in stress mode
Summary: [ews] mac-wk2 build marked as green even though it found failure in stress mode
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Aakash Jain
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2023-08-10 03:36 PDT by Aakash Jain
Modified: 2023-08-16 08:32 PDT (History)
5 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Aakash Jain 2023-08-10 03:36:07 PDT 
[ews] mac-wk2 build marked as green even though it found stress test failure.

In mac-wk2 (and similar queues) we run both: 1) test-being modified in stress mode, 2) complete layout-test suites.

Ideally if anyone detects a failure, ews bubble should be marked as red.

However, it seems to be marked as green when complete layout-test run was successful (failures were flaky).

e.g.: 
https://ews-build.webkit.org/#/builders/36/builds/13373
https://ews-build.webkit.org/#/builders/10/builds/13207
for PR https://github.com/WebKit/WebKit/pull/16505 (hash e5920c3)

Note that we also have a separate stress queue which had red bubble because of this failure (https://ews-build.webkit.org/#/builders/8/builds/13063). So, the issue wasn't completely hidden.
Comment 1 Aakash Jain 2023-08-11 10:54:12 PDT 
Pull request: https://github.com/WebKit/WebKit/pull/16611
Comment 2 EWS 2023-08-11 11:54:44 PDT 
Committed 266824@main (ad0590629fe0): <https://commits.webkit.org/266824@main>

Reviewed commits have been landed. Closing PR #16611 and removing active labels.
Comment 3 Radar WebKit Bug Importer 2023-08-11 11:55:14 PDT 
<rdar://problem/113763261>
Comment 4 Aakash Jain 2023-08-16 08:32:30 PDT 
This change is working fine. 
e.g.: https://ews-build.webkit.org/#/builders/10/builds/13514 is marked as red now (with GitHub status-bubble also red in https://github.com/WebKit/WebKit/pull/16658#issuecomment-1676773405).