Bug 185090 - REGRESSION (r230998): Cannot stream API test output
Summary: REGRESSION (r230998): Cannot stream API test output
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Jonathan Bedard
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2018-04-27 14:08 PDT by Jonathan Bedard
Modified: 2018-05-10 17:43 PDT (History)
6 users (show)

See Also:


Attachments
Patch (7.91 KB, patch)
2018-04-27 14:11 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (16.02 KB, patch)
2018-04-30 10:14 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch for landing (16.09 KB, patch)
2018-05-10 16:08 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Bedard 2018-04-27 14:08:14 PDT
It used to be the case that API tests would stream their output before timing out.  After r230998, this output was cached and only printed after a test timed out.
Comment 1 Jonathan Bedard 2018-04-27 14:11:24 PDT
Created attachment 339025 [details]
Patch
Comment 2 Ryosuke Niwa 2018-04-27 14:40:43 PDT
Comment on attachment 339025 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=339025&action=review

> Tools/Scripts/webkitpy/api_tests/runner.py:108
> +                        self.printer.writeln(source)

Does this output with timestamps? I really don't want to see that in the streamed results.
Because it would make copying & pasting multiple lines of output hard.
e.g. for dumping render tree, etc...

> Tools/Scripts/webkitpy/api_tests/runner.py:159
> +        status = Runner.STATUS_FAILED

Sounds like this is really STATUS_RUNNING?

> Tools/Scripts/webkitpy/api_tests/runner.py:166
> +            if status != Runner.STATUS_DISABLED:
> +                server_process.start()

Why is this code change needed?

> Tools/Scripts/webkitpy/api_tests/runner.py:177
> +                    self._caller.post('log', None, None, stderr_line[:-1])

Name arguments to which you're specifying None. Otherwise, the reader of this code can't tell what's being set to None.

> Tools/Scripts/webkitpy/api_tests/runner.py:183
> +                        self._caller.post('log', None, None, stdout_line[:-1])

Ditto.
Comment 3 Ryosuke Niwa 2018-04-27 14:41:41 PDT
Comment on attachment 339025 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=339025&action=review

> Tools/Scripts/webkitpy/api_tests/runner.py:52
> +        self._has_logged_for_test = True

Why this instance variable initially set to True?
Comment 4 Jonathan Bedard 2018-04-30 09:32:53 PDT
Comment on attachment 339025 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=339025&action=review

>> Tools/Scripts/webkitpy/api_tests/runner.py:52
>> +        self._has_logged_for_test = True
> 
> Why this instance variable initially set to True?

This is basically a cosmetic difference.

False would mean an empty line between the output 'Running tests', true means there is no empty line. I thought that not having the empty line looked better.

>> Tools/Scripts/webkitpy/api_tests/runner.py:108
>> +                        self.printer.writeln(source)
> 
> Does this output with timestamps? I really don't want to see that in the streamed results.
> Because it would make copying & pasting multiple lines of output hard.
> e.g. for dumping render tree, etc...

This would output timestamps in verbose mode, yes.

It sounds like that isn't desirable, I'll change it.

>> Tools/Scripts/webkitpy/api_tests/runner.py:166
>> +                server_process.start()
> 
> Why is this code change needed?

Because we really shouldn't have been starting the process if the test was disabled in the first place.

This code change isn't needed, exactly (the old way did work), but it is better because it doesn't spawn an extra process needlessly.
Comment 5 Jonathan Bedard 2018-04-30 10:14:54 PDT
Created attachment 339123 [details]
Patch
Comment 6 Jonathan Bedard 2018-04-30 10:21:52 PDT
Comment on attachment 339123 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=339123&action=review

> Tools/Scripts/webkitpy/api_tests/runner.py:100
> +        source = '' if self._num_workers == 1 else source + ' '

I wanted to call out what changed here, base on Ryosuke's request in comment 2: 'Does this output with timestamps? I really don't want to see that in the streamed results.'

The new patch will not print timestamps unless specifically asked to do so. It will also not print the worker information if only one worker is used. This should make logs easy to copy paste.
Comment 7 Ryosuke Niwa 2018-05-10 14:35:46 PDT
Comment on attachment 339123 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=339123&action=review

> Tools/Scripts/webkitpy/api_tests/runner.py:53
> +        self._has_logged_for_test = True

We should add a comment here saying this would suppress an empty line between "Running tests" and the first test's output.

> Tools/Scripts/webkitpy/layout_tests/views/metered_stream.py:57
> +        self._print_timestamps = verbose if print_timestamps is None else print_timestamps

It's a bit odd that this falls back to verbose but I guess changing it everywhere would be a lot of code changes.
Comment 8 Jonathan Bedard 2018-05-10 16:08:35 PDT
Created attachment 340142 [details]
Patch for landing
Comment 9 WebKit Commit Bot 2018-05-10 17:42:08 PDT
Comment on attachment 340142 [details]
Patch for landing

Clearing flags on attachment: 340142

Committed r231679: <https://trac.webkit.org/changeset/231679>
Comment 10 WebKit Commit Bot 2018-05-10 17:42:10 PDT
All reviewed patches have been landed.  Closing bug.
Comment 11 Radar WebKit Bug Importer 2018-05-10 17:43:43 PDT
<rdar://problem/40148803>