Bug 36078

Summary: new-run-webkit-tests fails java/lc3 on a clean checkout
Product: WebKit Reporter: Dirk Pranke <dpranke>
Component: Tools / TestsAssignee: Eric Seidel (no email) <eric>
Status: RESOLVED FIXED    
Severity: Normal CC: abarth, ap, cjerdonek, commit-queue, eric
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: PC   
OS: OS X 10.5   
Attachments:
Description Flags
Patch abarth: review+, commit-queue: commit-queue-

Description Dirk Pranke 2010-03-12 17:06:47 PST
Apparently someone recently introduced a java class for some tests and run-webkit-tests compiles this class as part of the test setup, but new-run-webkit-tests does not. So, all of the tests under java/lc3 will fail.
Comment 1 Alexey Proskuryakov 2010-03-12 22:45:09 PST
Do only java/lc3 tests fail? Other ones in java directory also need to compile java sources.
Comment 2 Eric Seidel (no email) 2010-03-26 12:16:39 PDT
I ran new-run-webkit-tests again last night.  Wow.  Very impressive.

But I ran into this bug.

if (isAppleMacWebKit()) {
    print STDERR "Compiling Java tests\n";
    my $javaTestsDirectory = catdir($testDirectory, "java");

    if (system("/usr/bin/make", "-C", "$javaTestsDirectory")) {
        exit 1;
    }
}

Seems to be the perl code used for the fix.  I'll prepare a patch.
Comment 3 Eric Seidel (no email) 2010-03-26 13:12:32 PDT
Created attachment 51774 [details]
Patch
Comment 4 Eric Seidel (no email) 2010-03-26 13:13:27 PDT
Comment on attachment 51774 [details]
Patch

I did not do a full new-run-webkit-tests run with this change.  My laptop takes an hour to build webkit.  I did however verify that the build part seems to be working. :)
Comment 5 Dirk Pranke 2010-03-26 13:16:57 PDT
Comment on attachment 51774 [details]
Patch

LGTM (although I'm not a reviewer).
Comment 6 Adam Barth 2010-03-26 13:23:02 PDT
Comment on attachment 51774 [details]
Patch

ok
Comment 7 Alexey Proskuryakov 2010-03-26 13:29:03 PDT
You didn't port this though:

    print STDERR "Compiling Java tests\n";

Does new-run-webkit-tests build quietly?
Comment 8 Eric Seidel (no email) 2010-03-26 13:35:40 PDT
new-run-webkit-tests in its default mode does everything quietly.  (Using one line which overwrites itself until it finally spits out results.)  I expect we'll eventually customize the logging to the greater webkit's whims and desires. :)

I don't know enough about all the various logging modes to know how to spit that part out in only the right modes.

Dirk would know more.
Comment 9 Dirk Pranke 2010-03-26 14:00:33 PDT
new-run-webkit-tests uses the "MeteredStream" class to provide the one-line status updates. The Port class doesn't actually get passed the handle to the meter, but it would be a good enhancement if it did.

I'll add that and add separate lines for the build steps in a different patch.

The general rule of thumbs I'm trying to follow for output are:

(1) By default, if everything works, we produce one line of output ("all tests completed successfully")

(2) Status updates are provided for every step that takes more than a second or two to run (so that you at least know what stage we're in). Only the current stage is displayed.

(3) Where possible, long-running stages actually produce a progress-bar style update (which is what meter.update() implements), so that you can tell that things aren't hung. Where possible, updates happen every few seconds or more frequently, up to ~10 times per second for a smooth animation). Currently only the actual running of the tests does this.
 
(I should probably document some of this stuff somewhere ...)
Comment 10 WebKit Commit Bot 2010-03-26 21:13:12 PDT
Comment on attachment 51774 [details]
Patch

Rejecting patch 51774 from commit-queue.

Failed to run "['/Users/eseidel/Projects/CommitQueue/WebKitTools/Scripts/svn-apply', '--reviewer', 'Adam Barth', '--force']" exit_code: 1
patching file WebKitTools/ChangeLog
Hunk #1 succeeded at 1 with fuzz 3.
patching file WebKitTools/Scripts/webkitpy/layout_tests/port/mac.py
Hunk #1 succeeded at 77 (offset 1 line).
Hunk #2 FAILED at 85.
1 out of 2 hunks FAILED -- saving rejects to file WebKitTools/Scripts/webkitpy/layout_tests/port/mac.py.rej

Full output: http://webkit-commit-queue.appspot.com/results/1578005
Comment 11 Eric Seidel (no email) 2010-03-29 11:39:47 PDT
Attachment 51774 [details] was posted by a committer and has review+, assigning to Eric Seidel for commit.
Comment 12 Eric Seidel (no email) 2010-03-29 15:21:05 PDT
Committed r56743: <http://trac.webkit.org/changeset/56743>