Bug 93563 – nrwt: should log progress like ninja does

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Dirk Pranke]

nrwt: should log progress like ninja does

[Dirk Pranke]

Created attachment 157354 [details]
Patch

[Ojan Vafai]

Comment on attachment 157354 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=157354&action=review

Can you post a few lines of the --verbose output?

> Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner_unittest.py:48
> +    nuM_tests = 0

typo

> Tools/Scripts/webkitpy/layout_tests/views/printing.py:348
> +                suffix += ' unexpectedly' + desc[2]

I know it's technically correct to say "unexpectedly" here...but I'm not sure it really adds valuable information in practice. I don't feel strongly, but I'd probably leave it out and see if people are confused.

[Dirk Pranke]

(In reply to comment #2)
> (From update of attachment 157354 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=157354&action=review
> 
> Can you post a few lines of the --verbose output?

Sure:

Scripts $ ./run-webkit-tests --chromium --verbose fast/html
Using port 'chromium-mac-lion'
Test configuration: <lion, x86, release>
Placing test results in None
Baseline search path: chromium-mac -> chromium -> mac -> generic
Using Release build
Pixel tests enabled
Regular timeout: 6000, slow test timeout: 30000
Command line: /src/wk/out/Release/DumpRenderTree.app/Contents/MacOS/DumpRenderTree -

Found 114 tests; running 114, skipping 0.
Running 1 DumpRenderTree over 1 shard.

[1/114] fast/html/adjacent-html-context-element.html passed
[2/114] fast/html/adopt-parent-frame.html passed

# ...

[113/114] fast/html/unknown-tag.html passed
[114/114] fast/html/xhtml-serialize.html passed

All 114 tests ran as expected.

%
 
> 
> > Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner_unittest.py:48
> > +    nuM_tests = 0
> 
> typo
>

will fix.
 
> > Tools/Scripts/webkitpy/layout_tests/views/printing.py:348
> > +                suffix += ' unexpectedly' + desc[2]
> 
> I know it's technically correct to say "unexpectedly" here...but I'm not sure it really adds valuable information in practice. I don't feel strongly, but I'd probably leave it out and see if people are confused.

If you don't print something to indicate expected/unexpected, then in --verbose mode you can't tell them apart, and that seems bad.

Also, note that the only real difference between the default logging and --verbose is that expected results get erased by default (so we're still printing "foo.html passed" or "foo.html failed (text diff)" for expected failures by default, that just gets erased by the next test.

Given your feedback, I tried the thought experiment of printing "as expected" for expected passes or failures, but that felt worse.

[Dirk Pranke]

Created attachment 157547 [details]
fix typo in layout_test_runner_unittest.py

[Ojan Vafai]

Comment on attachment 157547 [details]
fix typo in layout_test_runner_unittest.py

Looks great. I think this + the config printing is a big improvement!

[Dirk Pranke]

Committed r125204: <http://trac.webkit.org/changeset/125204>