Bug 184997

Summary:	run-api-tests summary included "LEAK" lines in list of test failures
Product:	WebKit	Reporter:	Ryan Haddad <ryanhaddad>
Component:	Tools / Tests	Assignee:	Nobody <webkit-unassigned>
Status:	RESOLVED WONTFIX
Severity:	Normal	CC:	jbedard, lforschler
Priority:	P2
Version:	Other
Hardware:	Unspecified
OS:	Unspecified

Description Ryan Haddad 2018-04-25 13:37:26 PDT

As seen below, run-api-tests summary included "LEAK" lines in list of test failures:
https://build.webkit.org/builders/Apple%20Sierra%20Debug%20WK1%20%28Tests%29/builds/7399/

12:43:34.053 96475 Test suite failed
12:43:34.053 96475 
12:43:34.054 96475 Failed
12:43:34.054 96475 
12:43:34.054 96475     TestWebKitAPI.WebKit.AutoLayoutIntegration
12:43:34.054 96475         
12:43:34.054 96475         /Volumes/Data/slave/sierra-debug/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/AutoLayoutIntegration.mm:98
12:43:34.054 96475         Value of: intrinsicContentSize.height
12:43:34.054 96475           Actual: 8
12:43:34.054 96475         Expected: _expectedIntrinsicContentSize.height
12:43:34.054 96475         Which is: 10
12:43:34.054 96475         
12:43:34.054 96475         
12:43:34.054 96475         /Volumes/Data/slave/sierra-debug/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/AutoLayoutIntegration.mm:97
12:43:34.054 96475         Value of: intrinsicContentSize.width
12:43:34.054 96475           Actual: 50
12:43:34.054 96475         Expected: _expectedIntrinsicContentSize.width
12:43:34.054 96475         Which is: 100
12:43:34.054 96475         
12:43:34.054 96475         
12:43:34.054 96475         /Volumes/Data/slave/sierra-debug/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/AutoLayoutIntegration.mm:98
12:43:34.054 96475         Value of: intrinsicContentSize.height
12:43:34.054 96475           Actual: 10
12:43:34.054 96475         Expected: _expectedIntrinsicContentSize.height
12:43:34.054 96475         Which is: 100
12:43:34.054 96475         
12:43:34.054 96475         
12:43:34.054 96475         /Volumes/Data/slave/sierra-debug/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/AutoLayoutIntegration.mm:98
12:43:34.054 96475         Value of: intrinsicContentSize.height
12:43:34.054 96475           Actual: 100
12:43:34.054 96475         Expected: _expectedIntrinsicContentSize.height
12:43:34.054 96475         Which is: 10
12:43:34.054 96475         
12:43:34.054 96475         
12:43:34.054 96475         /Volumes/Data/slave/sierra-debug/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/AutoLayoutIntegration.mm:97
12:43:34.054 96475         Value of: intrinsicContentSize.width
12:43:34.054 96475           Actual: 20
12:43:34.054 96475         Expected: _expectedIntrinsicContentSize.width
12:43:34.054 96475         Which is: 100
12:43:34.054 96475         
12:43:34.054 96475         
12:43:34.054 96475         /Volumes/Data/slave/sierra-debug/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/AutoLayoutIntegration.mm:98
12:43:34.054 96475         Value of: intrinsicContentSize.height
12:43:34.054 96475           Actual: 50
12:43:34.054 96475         Expected: _expectedIntrinsicContentSize.height
12:43:34.054 96475         Which is: 110
12:43:34.054 96475         
12:43:34.054 96475         LEAK: 53 BidiRun
12:43:34.055 96475         LEAK: 14 WebCoreNode
12:43:34.055 96475

Comment 1 Jonathan Bedard 2018-04-25 16:02:00 PDT

This was a deliberate change.

We capture the STDOUT and STDERR of the test, this includes the record of leaks.

It's possible to filter these out, is that desirable?

Comment 2 Ryan Haddad 2018-04-25 16:12:11 PDT

(In reply to Jonathan Bedard from comment #1)
> This was a deliberate change.
> 
> We capture the STDOUT and STDERR of the test, this includes the record of
> leaks.
> 
> It's possible to filter these out, is that desirable?
Ah, I see. I am indifferent, so if they are useful for someone they should stay.

Comment 3 Jonathan Bedard 2018-04-25 16:22:02 PDT

They would be meaningful if we fixed some of the underlying test runner bugs.

Actually, they are indicative of a memory leak similar to a pretty serious memory leak we saw in the Layout test runner a few months back.  I would be in favor of keeping these, especially with the verbose flag.