Bug 93563 - nrwt: should log progress like ninja does
Summary: nrwt: should log progress like ninja does
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Dirk Pranke
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-08-08 17:52 PDT by Dirk Pranke
Modified: 2012-08-09 14:36 PDT (History)
7 users (show)

See Also:


Attachments
Patch (10.64 KB, patch)
2012-08-08 18:02 PDT, Dirk Pranke
no flags Details | Formatted Diff | Diff
fix typo in layout_test_runner_unittest.py (10.64 KB, patch)
2012-08-09 14:24 PDT, Dirk Pranke
ojan: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Dirk Pranke 2012-08-08 17:52:31 PDT
nrwt: should log progress like ninja does
Comment 1 Dirk Pranke 2012-08-08 18:02:13 PDT
Created attachment 157354 [details]
Patch
Comment 2 Ojan Vafai 2012-08-09 14:10:45 PDT
Comment on attachment 157354 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=157354&action=review

Can you post a few lines of the --verbose output?

> Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner_unittest.py:48
> +    nuM_tests = 0

typo

> Tools/Scripts/webkitpy/layout_tests/views/printing.py:348
> +                suffix += ' unexpectedly' + desc[2]

I know it's technically correct to say "unexpectedly" here...but I'm not sure it really adds valuable information in practice. I don't feel strongly, but I'd probably leave it out and see if people are confused.
Comment 3 Dirk Pranke 2012-08-09 14:19:01 PDT
(In reply to comment #2)
> (From update of attachment 157354 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=157354&action=review
> 
> Can you post a few lines of the --verbose output?

Sure:

Scripts $ ./run-webkit-tests --chromium --verbose fast/html
Using port 'chromium-mac-lion'
Test configuration: <lion, x86, release>
Placing test results in None
Baseline search path: chromium-mac -> chromium -> mac -> generic
Using Release build
Pixel tests enabled
Regular timeout: 6000, slow test timeout: 30000
Command line: /src/wk/out/Release/DumpRenderTree.app/Contents/MacOS/DumpRenderTree -

Found 114 tests; running 114, skipping 0.
Running 1 DumpRenderTree over 1 shard.

[1/114] fast/html/adjacent-html-context-element.html passed
[2/114] fast/html/adopt-parent-frame.html passed

# ...

[113/114] fast/html/unknown-tag.html passed
[114/114] fast/html/xhtml-serialize.html passed

All 114 tests ran as expected.

%
 
> 
> > Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner_unittest.py:48
> > +    nuM_tests = 0
> 
> typo
>

will fix.
 
> > Tools/Scripts/webkitpy/layout_tests/views/printing.py:348
> > +                suffix += ' unexpectedly' + desc[2]
> 
> I know it's technically correct to say "unexpectedly" here...but I'm not sure it really adds valuable information in practice. I don't feel strongly, but I'd probably leave it out and see if people are confused.

If you don't print something to indicate expected/unexpected, then in --verbose mode you can't tell them apart, and that seems bad.

Also, note that the only real difference between the default logging and --verbose is that expected results get erased by default (so we're still printing "foo.html passed" or "foo.html failed (text diff)" for expected failures by default, that just gets erased by the next test.

Given your feedback, I tried the thought experiment of printing "as expected" for expected passes or failures, but that felt worse.
Comment 4 Dirk Pranke 2012-08-09 14:24:54 PDT
Created attachment 157547 [details]
fix typo in layout_test_runner_unittest.py
Comment 5 Ojan Vafai 2012-08-09 14:27:47 PDT
Comment on attachment 157547 [details]
fix typo in layout_test_runner_unittest.py

Looks great. I think this + the config printing is a big improvement!
Comment 6 Dirk Pranke 2012-08-09 14:36:04 PDT
Committed r125204: <http://trac.webkit.org/changeset/125204>