Bug 63439 - new-run-webkit-tests needs a --webkit-test-runner option
Summary: new-run-webkit-tests needs a --webkit-test-runner option
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Eric Seidel (no email)
URL:
Keywords:
Depends on: 63476
Blocks: nrwt-wk2
  Show dependency treegraph
 
Reported: 2011-06-27 01:48 PDT by Eric Seidel (no email)
Modified: 2011-06-28 14:44 PDT (History)
3 users (show)

See Also:


Attachments
Patch (23.52 KB, patch)
2011-06-27 01:54 PDT, Eric Seidel (no email)
no flags Details | Formatted Diff | Diff
Patch for landing (25.32 KB, patch)
2011-06-27 14:40 PDT, Eric Seidel (no email)
no flags Details | Formatted Diff | Diff
Patch for landing (33.39 KB, patch)
2011-06-27 15:23 PDT, Eric Seidel (no email)
no flags Details | Formatted Diff | Diff
Patch (2.39 KB, patch)
2011-06-28 11:09 PDT, Eric Seidel (no email)
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Eric Seidel (no email) 2011-06-27 01:48:28 PDT
new-run-webkit-tests needs a --webkit-test-runner option
Comment 1 Eric Seidel (no email) 2011-06-27 01:54:56 PDT
Created attachment 98683 [details]
Patch
Comment 2 Adam Barth 2011-06-27 01:57:33 PDT
Comment on attachment 98683 [details]
Patch

Ok.
Comment 3 WebKit Review Bot 2011-06-27 10:42:34 PDT
The commit-queue encountered the following flaky tests while processing attachment 98683 [details]:

http/tests/local/formdata/send-form-data-constructed-from-form.html bug 62431 (author: jianli@chromium.org)
The commit-queue is continuing to process your patch.
Comment 4 WebKit Review Bot 2011-06-27 10:44:08 PDT
Comment on attachment 98683 [details]
Patch

Clearing flags on attachment: 98683

Committed r89834: <http://trac.webkit.org/changeset/89834>
Comment 5 WebKit Review Bot 2011-06-27 10:44:13 PDT
All reviewed patches have been landed.  Closing bug.
Comment 6 Adam Barth 2011-06-27 13:47:54 PDT
======================================================================
FAIL: test_build_driver (webkitpy.layout_tests.port.webkit_unittest.WebKitPortTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/abarth/svn/webkit/Tools/Scripts/webkitpy/layout_tests/port/webkit_unittest.py", line 124, in test_build_driver
    self.assertTrue(output.assert_outputs(self, port._build_driver, expected_stderr=expected_stderr))
  File "/Users/abarth/svn/webkit/Tools/Scripts/webkitpy/common/system/outputcapture.py", line 66, in assert_outputs
    testcase.assertEqual(stderr_string, expected_stderr)
AssertionError: "MOCK run_command: ['/Users/abarth/svn/webkit/Tools/Scripts/build-dumprendertree', '--release']\n" != "MOCK run_command: ['/Projects/WebKit/Tools/Scripts/build-dumprendertree', '--release']\n"

======================================================================
FAIL: test_run_singly_actually_runs_tests (webkitpy.layout_tests.run_webkit_tests_integrationtest.MainTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/abarth/svn/webkit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py", line 351, in test_run_singly_actually_runs_tests
    self.assertEquals(res, 5)
AssertionError: 6 != 5

======================================================================
FAIL: test_unexpected_failures (webkitpy.layout_tests.run_webkit_tests_integrationtest.MainTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/abarth/svn/webkit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py", line 403, in test_unexpected_failures
    self.assertEqual(res, unexpected_tests_count)
AssertionError: 6 != 5

----------------------------------------------------------------------
Ran 1035 tests in 25.396s

FAILED (failures=3)
Comment 7 Dirk Pranke 2011-06-27 13:56:18 PDT
Comment on attachment 98683 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=98683&action=review

> Tools/Scripts/webkitpy/layout_tests/port/config.py:107
> +    # if scm.find_checkout_root() is broken for Chromium, we should fix (or at least wrap) it!

scm.find_checkout_root() is a generic routine to find the top of a given VCS's checkout. It does not (ostensibly) have anything to do with the actual top of the webkit tree, since it can be reused in other non-webkit checkouts (this is what I was told by you and Adam about the intent of the scm module, at least).

If you want the checkout module to be webkit-specific, then it would be great if that had an equivalent to config.webkit_base_dir. 

FWIW, the whole config module should really be moved into webkitpy/common somewhere, since none of it is layout-test-specific.

Note that any routine that would replace this one has to support at least two uses that Chromium needs:

1) Working in a bare tree (one that doesn't have .git or .svn metadata directories); this is a requested feature for the chromium test bots

2) Working in a tree where the top of the webkit directory != the top of the checkout, and where you may not have everything in the checkout present (in Chromium, for example, they check out the Source and LayoutTests directories directly, rather than checking out the whole tree).

> Tools/Scripts/webkitpy/layout_tests/port/webkit_unittest.py:-114
> -    unittest.main()

Why did you delete these lines? I frequently run tests of individual modules this way, and would prefer that they stay in the files.

> Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py:271
> +            help="Run a concurrent JavaScript thead with each test"),

Nit. Typo: "thread", not "thead"

> Tools/Scripts/webkitpy/tool/bot/queueengine_unittest.py:163
> +        OutputCapture().assert_outputs(self, engine.run, expected_stderr=expected_stderr)

Did this have anything to do with the other changes?

> Tools/Scripts/webkitpy/tool/commands/commandtest.py:35
> +    def assert_execute_outputs(self, command, args=[], expected_stdout="", expected_stderr="", expected_exception=None, options=MockOptions(), tool=MockTool()):

Ditto.

> Tools/Scripts/webkitpy/tool/commands/queues_unittest.py:169
> +        patch_id = OutputCapture().assert_outputs(self, queue._next_patch, expected_stdout=expected_stdout, expected_stderr=expected_stderr)

Ditto.
Comment 8 Eric Seidel (no email) 2011-06-27 14:40:37 PDT
Created attachment 98787 [details]
Patch for landing
Comment 9 WebKit Review Bot 2011-06-27 14:49:56 PDT
Comment on attachment 98787 [details]
Patch for landing

Rejecting attachment 98787 [details] from commit-queue.

Failed to run "['./Tools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=ec2-cq-01', '--port..." exit_code: 1

Last 500 characters of output:
139fd5220d9bbbf17b9ffc07a24c553015a8c25f
r89863 = 97c078b725714bdefe986a4ed07e31d1f9e900ab
Done rebuilding .git/svn/refs/remotes/origin/master/.rev_map.268f45cc-cd09-0410-ab3c-d52691b4dbfc
First, rewinding head to replay your work on top of it...
Fast-forwarded master to refs/remotes/origin/master.
Updating chromium port dependencies using gclient...

________ running '/usr/bin/python gyp_webkit' in '/mnt/git/webkit-commit-queue/Source/WebKit/chromium'
Updating webkit projects from gyp files...

Full output: http://queues.webkit.org/results/8948426
Comment 10 WebKit Review Bot 2011-06-27 15:17:58 PDT
Comment on attachment 98787 [details]
Patch for landing

Clearing flags on attachment: 98787

Committed r89868: <http://trac.webkit.org/changeset/89868>
Comment 11 WebKit Review Bot 2011-06-27 15:18:03 PDT
All reviewed patches have been landed.  Closing bug.
Comment 12 Eric Seidel (no email) 2011-06-27 15:23:57 PDT
Created attachment 98801 [details]
Patch for landing
Comment 13 Eric Seidel (no email) 2011-06-27 15:39:05 PDT
webkit-patch should auto-open bugs when uploading new patches. :(
Comment 14 WebKit Review Bot 2011-06-27 15:40:53 PDT
Comment on attachment 98801 [details]
Patch for landing

Rejecting attachment 98801 [details] from commit-queue.

Failed to run "['./Tools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=ec2-cq-03', '--port..." exit_code: 2

Last 500 characters of output:
.rej
patching file Tools/Scripts/webkitpy/tool/commands/commandtest.py
Hunk #1 FAILED at 32.
1 out of 1 hunk FAILED -- saving rejects to file Tools/Scripts/webkitpy/tool/commands/commandtest.py.rej
patching file Tools/Scripts/webkitpy/tool/commands/queues_unittest.py
Hunk #1 FAILED at 166.
1 out of 1 hunk FAILED -- saving rejects to file Tools/Scripts/webkitpy/tool/commands/queues_unittest.py.rej

Failed to run "[u'/mnt/git/webkit-commit-queue/Tools/Scripts/svn-apply', u'--force']" exit_code: 1

Full output: http://queues.webkit.org/results/8954078
Comment 15 Eric Seidel (no email) 2011-06-27 16:03:10 PDT
Was already landed.
Comment 16 Ryosuke Niwa 2011-06-28 11:00:21 PDT
A python test is failing on Chromium Windows bots after this patch was landed:
http://build.webkit.org/builders/Chromium%20Win%20Release%20%28Tests%29/builds/16070/steps/webkitpy-test/logs/stdio

Traceback (most recent call last):
  File "E:\google-windows-2\chromium-win-release-tests\build\Tools\Scripts\webkitpy\layout_tests\port\webkit_unittest.py", line 124, in test_build_driver
    self.assertTrue(output.assert_outputs(self, port._build_driver, expected_stderr=expected_stderr))
  File "E:\google-windows-2\chromium-win-release-tests\build\Tools\Scripts\webkitpy\common\system\outputcapture.py", line 66, in assert_outputs
    testcase.assertEqual(stderr_string, expected_stderr)
AssertionError: "MOCK run_command: ['Tools\\\\Scripts\\\\build-dumprendertree', '--release']\n" != "MOCK run_command: ['Tools/Scripts/build-dumprendertree', '--release']\n"
Comment 17 Eric Seidel (no email) 2011-06-28 11:01:19 PDT
OK.  Easy fix, will write it now.
Comment 18 Eric Seidel (no email) 2011-06-28 11:06:56 PDT
reopen to fix windows.
Comment 19 Eric Seidel (no email) 2011-06-28 11:09:21 PDT
Created attachment 98944 [details]
Patch
Comment 20 Eric Seidel (no email) 2011-06-28 11:16:01 PDT
re-open to fix windows.
Comment 21 Eric Seidel (no email) 2011-06-28 13:41:25 PDT
(In reply to comment #20)
> re-open to fix windows.

I finally fixed webkit-patch to know how to re-open bugs when posting new patches! bug 32006.
Comment 22 WebKit Review Bot 2011-06-28 14:44:30 PDT
Comment on attachment 98944 [details]
Patch

Clearing flags on attachment: 98944

Committed r89962: <http://trac.webkit.org/changeset/89962>
Comment 23 WebKit Review Bot 2011-06-28 14:44:36 PDT
All reviewed patches have been landed.  Closing bug.