NRWT sometimes fails to identify timed-out tests (once?)

• Component: Tools / Tests
• Status: RESOLVED INVALID

[Tim Horton]

I've only seen this once so far, but I don't usually click through the "stderr output" column.

This run:

http://build.webkit.org/results/Apple%20Lion%20Release%20WK1%20(Tests)/r128059%20(3366)/results.html

Has no timeouts listed on the results page, but the stderr output for http/tests/images/jpg-img-partial-load.html:

http://build.webkit.org/results/Apple%20Lion%20Release%20WK1%20(Tests)/r128059%20(3366)/http/tests/images/jpg-img-partial-load-stderr.txt

Says:

"FAIL: Timed out waiting for notifyDone to be called"

Odd that this doesn't show up in the summary.

I don't this failures to call notifyDone are are considered timeouts (they are handled by an internal watchdog in the test runner, which prints this message and then moves on), but perhaps I'm wrong.

s/this/think/

[Tim Horton]

(In reply to comment #1)
> I don't this failures to call notifyDone are are considered timeouts (they are handled by an internal watchdog in the test runner, which prints this message and then moves on), but perhaps I'm wrong.

Hmm. A test that times out in this way should certainly be noted separately on the results page. Perhaps not as a timeout, but at least as a failure:

http://build.webkit.org/results/Apple%20Lion%20Release%20WK2%20(Tests)/r128059%20(3081)/results.html

http://build.webkit.org/results/Apple%20Lion%20Release%20WK2%20(Tests)/r128059%20(3081)/fast/history/history-back-initial-vs-final-url-pretty-diff.html

Or maybe I'm crazy. I don't think the stderr list usually "counts", though...

[Tim Horton]

Those are both dumpAsText, too, so that's not the issue.

If timing out didn't affect the test's output, one can't help but wonder why the test is using waitUntilDone.

[Tim Horton]

(In reply to comment #5)
> If timing out didn't affect the test's output, one can't help but wonder why the test is using waitUntilDone.

It looks like it's a pixel-only test. The -expected output is blank.

AHA. In TestExpectations:

// (r125185) http/tests/images/jpg-img-partial-load.html timeout on Lion
BUGWK93636 : http/tests/images/jpg-img-partial-load.html = TEXT

Sooooo... hmm. I guess this is not actually a problem. Would be nice if it didn't show up in the stderr list, though.