Bug 106688

Summary:

REGRESSION: buildbot mangles error messages with timestamps run-webkit-tests outputs

Product:

WebKit

Reporter:

Ryosuke Niwa <rniwa>

Component:

Tools / Tests

Assignee:

Ryosuke Niwa <rniwa>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

abarth, andersca, dpranke, eric, lforschler, ossy, simon.fraser, slewis, thorton

Priority:

Version:

528+ (Nightly build)

Hardware:

Unspecified

OS:

Unspecified

Attachments:

Description	Flags
Fixes the bug	none
Mac ML leaks bot screenshot (now with regression)	none
Mac ML leaks bot screenshot (with patch)	none
Updated the test	eric: review+

Ryosuke Niwa

Reported 2013-01-11 12:21:56 PST

When run-webkit-tests spits out error messages like: 12:04:45.699 18907 Exiting early after 6 crashes and 14 timeouts. 8872 tests run. We're supposed to be extracting "Exiting early after 6 crashes and 14 timeouts. 8872 tests run", ignoring the leading timestamp and whatever digit that comes after it.

Attachments
Fixes the bug (1.50 KB, patch) 2013-01-11 12:26 PST, Ryosuke Niwa	no flags	Details Formatted Diff Diff
Mac ML leaks bot screenshot (now with regression) (19.34 KB, image/png) 2013-01-11 12:36 PST, Ryosuke Niwa	no flags	Details
Mac ML leaks bot screenshot (with patch) (15.28 KB, image/png) 2013-01-11 12:39 PST, Ryosuke Niwa	no flags	Details
Updated the test (2.61 KB, patch) 2013-01-11 13:39 PST, Ryosuke Niwa	eric: review+	Details Formatted Diff Diff
Show Obsolete (1) View All Add attachment proposed patch, testcase, etc.

Ryosuke Niwa

Comment 1 2013-01-11 12:22:55 PST

This regressed when run-webkit-tests changed stdout format.

Ryosuke Niwa

Comment 2 2013-01-11 12:26:21 PST

Created attachment 182394 [details] Fixes the bug

Ryosuke Niwa

Comment 3 2013-01-11 12:28:17 PST

e.g. http://build.webkit.org/builders/GTK%20Linux%2064-bit%20Debug/builds/40707 Once this patch is landed, 11:09:27.593 11406 Exiting early after 0 crashes and 20 timeouts. 14935 tests run. 21 failures 1 new passes ( 31 mins, 47 secs ) should look like: Exiting early after 0 crashes and 20 timeouts. 14935 tests run. 21 failures 1 new passes ( 31 mins, 47 secs )

Ryosuke Niwa

Comment 4 2013-01-11 12:36:39 PST

Created attachment 182395 [details] Mac ML leaks bot screenshot (now with regression)

Ryosuke Niwa

Comment 5 2013-01-11 12:39:41 PST

Created attachment 182396 [details] Mac ML leaks bot screenshot (with patch)

Eric Seidel (no email)

Comment 6 2013-01-11 13:05:03 PST

I remember spending some time on this during the transision. I guess we didn't get every case.

Eric Seidel (no email)

Comment 7 2013-01-11 13:06:09 PST

The current regexp is clearly designed for NRWT, since ORWT didnt' have log channels. :) master.cfg has tests, which test-webkitpy is able to run, btw. We could test this.

Ryosuke Niwa

Comment 8 2013-01-11 13:39:59 PST

Created attachment 182409 [details] Updated the test

Ryosuke Niwa

Comment 9 2013-01-11 13:40:41 PST

These tests aren't useful because they don't use real outputs from run-webkit-tests.

Ryosuke Niwa

Comment 10 2013-01-11 13:46:05 PST

(In reply to comment #7) > The current regexp is clearly designed for NRWT, since ORWT didnt' have log channels. :) I wrote the said regular expression to ignore the prefix of NRWT logs to be used by the parser for NRWT. As the name suggests, this regular expression isn't used for ORWT; for that matter, ORWT doesn't need us stripping prefix like this because it doesn't have one.

Eric Seidel (no email)

Comment 11 2013-01-11 14:00:30 PST

Comment on attachment 182409 [details] Updated the test I would expect we could use real NRWT classes to produce real logs if that's what you were interested in testing.

Ryosuke Niwa

Comment 12 2013-01-11 14:58:43 PST

Committed r139505: <http://trac.webkit.org/changeset/139505>

Note You need to log in before you can comment on or make changes to this bug.