RESOLVED DUPLICATE of bug 62178 59894
waterfall or console doesn't report the number of failures for new-run-webkit-tests 
https://bugs.webkit.org/show_bug.cgi?id=59894
Summary waterfall or console doesn't report the number of failures for new-run-webkit...
Ryosuke Niwa
Reported 2011-05-01 14:22:13 PDT
For old-run-webkit-tests, build.webkit.org/waterfall reports how many tests are failed. However, such information isn't available for bots that run new-run-webkit-tests. See http://build.webkit.org/builders/SnowLeopard%20Intel%20Release%20%28WebKit2%20Tests%29/builds/11263 http://build.webkit.org/builders/Leopard%20Intel%20Release%20%28NRWT%29/builds/70
Attachments
Add attachment proposed patch, testcase, etc.
Tony Chang
Comment 1 2011-05-02 10:31:47 PDT
The number comes from parsing the std output of the script. Since NRWT has different output, there's no number. We could copy parser used by the chromium bots (that would also give us other stuff like listing the failing tests and linking to the flakiness dashboard).
Ryosuke Niwa
Comment 2 2011-05-02 11:11:14 PDT
(In reply to comment #1) > The number comes from parsing the std output of the script. Since NRWT has different output, there's no number. > > We could copy parser used by the chromium bots (that would also give us other stuff like listing the failing tests and linking to the flakiness dashboard). I find the number of failures to be much more useful than a partial listing of failed list.
Tony Chang
Comment 3 2011-05-02 11:19:50 PDT
(In reply to comment #2) > (In reply to comment #1) > > We could copy parser used by the chromium bots (that would also give us other stuff like listing the failing tests and linking to the flakiness dashboard). > > I find the number of failures to be much more useful than a partial listing of failed list. This is not an either-or situation :) The Chromium bots have both the number of failures and the list of failing tests. Perhaps you're saying that listing the failing tests makes it harder to scan and see the failed number? I think abarth was advocating having the list of failing tests so you don't have to make 2 more clicks (results dir + results.html) to figure out which test failed. This could be done as a separate patch.
Ryosuke Niwa
Comment 4 2011-05-02 11:35:54 PDT
(In reply to comment #3) > This is not an either-or situation :) The Chromium bots have both the number of failures and the list of failing tests. Perhaps you're saying that listing the failing tests makes it harder to scan and see the failed number? Yeah. When there are many tests failing (20+), it's useful to have a number of tests failing so that when I skim through the list of builds, I can easily find where the number increased. > I think abarth was advocating having the list of failing tests so you don't have to make 2 more clicks (results dir + results.html) to figure out which test failed. This could be done as a separate patch. I think the list of failures will be useful when there are less than ~5 test failures.
Ryosuke Niwa
Comment 5 2011-06-06 23:56:13 PDT
Oops, I forgot about this bug. Merging it to the bug 62178 since it already has a patch. *** This bug has been marked as a duplicate of bug 62178 ***
Note You need to log in before you can comment on or make changes to this bug.