Bug 206545 - PerformanceTests should not treat stderr output as a test failure
Summary: PerformanceTests should not treat stderr output as a test failure
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-01-21 11:41 PST by Brent Fulgham
Modified: 2020-01-21 23:38 PST (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Brent Fulgham 2020-01-21 11:41:56 PST
The PerformanceTests test harness treats any stderr output as a test failure. Instead, it should treat it like the normal test system, where stderr is captured and noted, but does not contribute to the pass or fail status of a test.
Comment 1 Radar WebKit Bug Importer 2020-01-21 11:42:09 PST
<rdar://problem/58767355>
Comment 2 Alexey Proskuryakov 2020-01-21 12:39:20 PST
IIRC Ryosuke found it very important to catch errors on this way. Perhaps this is less relevant after redoing much of the logging system?
Comment 3 Ryosuke Niwa 2020-01-21 23:38:17 PST
(In reply to Brent Fulgham from comment #0)
> The PerformanceTests test harness treats any stderr output as a test
> failure. Instead, it should treat it like the normal test system, where
> stderr is captured and noted, but does not contribute to the pass or fail
> status of a test.

I don't think we want to do this. Some of our internal benchmarks get broken all the time because we ignore all error messages.