Bug 155476 – run-api-tests doesn't print test name when the test crashes

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Alexey Proskuryakov]

When run-api-tests crashes, the test name is not printed. Even though the test is counted as a failing one and printed at the end, this makes it unnecessarily hard to find assertion stack traces in run-api-tests output.

[Alexey Proskuryakov]

Created attachment 274057 [details]
proposed fix

[Build Bot]

Comment on attachment 274057 [details]
proposed fix

Attachment 274057 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/979670

New failing tests:
js/function-call-aliased.html

[Build Bot]

Created attachment 274061 [details]
Archive of layout-test-results from ews114 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews114  Port: mac-yosemite  Platform: Mac OS X 10.10.5

[Daniel Bates]

Comment on attachment 274057 [details]
proposed fix

View in context: https://bugs.webkit.org/attachment.cgi?id=274057&action=review

> Tools/Scripts/run-api-tests:237
> +            if (!$timedOut && index("@testOutput", $test) == -1) {
> +                print STDOUT possiblyColored("bold red", "UNEXPECTEDLY EXITED"), " $test\n";
> +            }

This is a circuitous way of determining that $test is not in the array @testOutput. We should make use of grep [1] to write this more directly:

if (!$timedOut && !grep { $_ eq $test } @testOutput) {
    print STDOUT possiblyColored("bold red", "UNEXPECTEDLY EXITED"), " $test\n";
}

[1] <http://perldoc.perl.org/functions/grep.html>

[Alexey Proskuryakov]

Is that indeed the case? I believe that it's an array of lines, and the test name doesn't take the whole line.

[Daniel Bates]

(In reply to comment #5)
> Is that indeed the case? I believe that it's an array of lines, and the test
> name doesn't take the whole line.

Notice that grep can we used with a regular expression. Using the \Q and \E specifiers to disable integration of regular expression metacharacters, can we write something of the form:

if (!$timedOut && !grep { /\Q$test\E/ } @testOutput) {
    print STDOUT possiblyColored("bold red", "UNEXPECTEDLY EXITED"), " $test\n";
}

We should look to strengthen this regular expression if possible.

[Daniel Bates]

(In reply to comment #6)
> (In reply to comment #5)
> > Is that indeed the case? I believe that it's an array of lines, and the test
> > name doesn't take the whole line.
> 
> Notice that grep can we used with a regular expression. Using the \Q and \E
> specifiers to disable integration of regular expression metacharacters 

I meant to write:

Notice that grep can be used with a regular expression. Using the metaquoting escape sequence \Q...\E (*) to disable interpretation of regular expression metacharacters, can we write something ...

(*) See <http://perldoc.perl.org/perlre.html#Quoting-metacharacters> for more details.

[Alexey Proskuryakov]

Comment on attachment 274057 [details]
proposed fix

Marking cq+ per discussion with Dan - the potential improvement does not seem to be worth the effort.

[WebKit Commit Bot]

Comment on attachment 274057 [details]
proposed fix

Clearing flags on attachment: 274057

Committed r198231: <http://trac.webkit.org/changeset/198231>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.