Bug 207118 - run-javascriptcore-tests: Don't split binary results
Summary: run-javascriptcore-tests: Don't split binary results
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Jonathan Bedard
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-02-03 08:28 PST by Jonathan Bedard
Modified: 2020-06-03 14:57 PDT (History)
5 users (show)

See Also:


Attachments
Patch (4.15 KB, patch)
2020-02-03 08:30 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Bedard 2020-02-03 08:28:51 PST
Experience over the last 2 months has shown that attempting to split up results from a single JSC test binary doesn't really work. We should just report aggregated results for a single binary.
Comment 1 Jonathan Bedard 2020-02-03 08:30:41 PST
Created attachment 389515 [details]
Patch
Comment 2 Jonathan Bedard 2020-02-03 08:32:15 PST
https://results.webkit.org/suites?suite=javascriptcore-tests demonstrates how my original approach basically failed.
Comment 3 Zhifei Fang 2020-02-03 10:13:36 PST
Comment on attachment 389515 [details]
Patch

unofficially r+, in long term we should unify all the test output format, and let those binaries generate that standard output
Comment 4 Aakash Jain 2020-02-03 10:23:34 PST
(In reply to Jonathan Bedard from comment #0)
> Experience over the last 2 months has shown that attempting to split up results from a single JSC test binary doesn't really work.
What's the issue?
Comment 5 Jonathan Bedard 2020-02-03 10:46:04 PST
(In reply to Aakash Jain from comment #4)
> (In reply to Jonathan Bedard from comment #0)
> > Experience over the last 2 months has shown that attempting to split up results from a single JSC test binary doesn't really work.
> What's the issue?

We can't really do it consistently, a sub-test which fails will sometimes have a different name than when it passes and often, exception messages will be tagged as actual test names.
Comment 6 Aakash Jain 2020-02-03 10:59:54 PST
rs=me
Comment 7 Jonathan Bedard 2020-02-03 11:37:43 PST
Committed r255578: <https://trac.webkit.org/changeset/255578>
Comment 8 Radar WebKit Bug Importer 2020-02-03 11:41:44 PST
<rdar://problem/59118504>