Bug 63439 – new-run-webkit-tests needs a --webkit-test-runner option

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Eric Seidel (no email)]

new-run-webkit-tests needs a --webkit-test-runner option

[Eric Seidel (no email)]

Created attachment 98683 [details]
Patch

[Adam Barth]

Comment on attachment 98683 [details]
Patch

Ok.

[WebKit Review Bot]

The commit-queue encountered the following flaky tests while processing attachment 98683 [details]:

http/tests/local/formdata/send-form-data-constructed-from-form.html bug 62431 (author: [e-mail])
The commit-queue is continuing to process your patch.

[WebKit Review Bot]

Comment on attachment 98683 [details]
Patch

Clearing flags on attachment: 98683

Committed r89834: <http://trac.webkit.org/changeset/89834>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.

[Adam Barth]

======================================================================
FAIL: test_build_driver (webkitpy.layout_tests.port.webkit_unittest.WebKitPortTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/abarth/svn/webkit/Tools/Scripts/webkitpy/layout_tests/port/webkit_unittest.py", line 124, in test_build_driver
    self.assertTrue(output.assert_outputs(self, port._build_driver, expected_stderr=expected_stderr))
  File "/Users/abarth/svn/webkit/Tools/Scripts/webkitpy/common/system/outputcapture.py", line 66, in assert_outputs
    testcase.assertEqual(stderr_string, expected_stderr)
AssertionError: "MOCK run_command: ['/Users/abarth/svn/webkit/Tools/Scripts/build-dumprendertree', '--release']\n" != "MOCK run_command: ['/Projects/WebKit/Tools/Scripts/build-dumprendertree', '--release']\n"

======================================================================
FAIL: test_run_singly_actually_runs_tests (webkitpy.layout_tests.run_webkit_tests_integrationtest.MainTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/abarth/svn/webkit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py", line 351, in test_run_singly_actually_runs_tests
    self.assertEquals(res, 5)
AssertionError: 6 != 5

======================================================================
FAIL: test_unexpected_failures (webkitpy.layout_tests.run_webkit_tests_integrationtest.MainTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/abarth/svn/webkit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py", line 403, in test_unexpected_failures
    self.assertEqual(res, unexpected_tests_count)
AssertionError: 6 != 5

----------------------------------------------------------------------
Ran 1035 tests in 25.396s

FAILED (failures=3)

[Dirk Pranke]

Comment on attachment 98683 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=98683&action=review

> Tools/Scripts/webkitpy/layout_tests/port/config.py:107
> +    # if scm.find_checkout_root() is broken for Chromium, we should fix (or at least wrap) it!

scm.find_checkout_root() is a generic routine to find the top of a given VCS's checkout. It does not (ostensibly) have anything to do with the actual top of the webkit tree, since it can be reused in other non-webkit checkouts (this is what I was told by you and Adam about the intent of the scm module, at least).

If you want the checkout module to be webkit-specific, then it would be great if that had an equivalent to config.webkit_base_dir. 

FWIW, the whole config module should really be moved into webkitpy/common somewhere, since none of it is layout-test-specific.

Note that any routine that would replace this one has to support at least two uses that Chromium needs:

1) Working in a bare tree (one that doesn't have .git or .svn metadata directories); this is a requested feature for the chromium test bots

2) Working in a tree where the top of the webkit directory != the top of the checkout, and where you may not have everything in the checkout present (in Chromium, for example, they check out the Source and LayoutTests directories directly, rather than checking out the whole tree).

> Tools/Scripts/webkitpy/layout_tests/port/webkit_unittest.py:-114
> -    unittest.main()

Why did you delete these lines? I frequently run tests of individual modules this way, and would prefer that they stay in the files.

> Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py:271
> +            help="Run a concurrent JavaScript thead with each test"),

Nit. Typo: "thread", not "thead"

> Tools/Scripts/webkitpy/tool/bot/queueengine_unittest.py:163
> +        OutputCapture().assert_outputs(self, engine.run, expected_stderr=expected_stderr)

Did this have anything to do with the other changes?

> Tools/Scripts/webkitpy/tool/commands/commandtest.py:35
> +    def assert_execute_outputs(self, command, args=[], expected_stdout="", expected_stderr="", expected_exception=None, options=MockOptions(), tool=MockTool()):

Ditto.

> Tools/Scripts/webkitpy/tool/commands/queues_unittest.py:169
> +        patch_id = OutputCapture().assert_outputs(self, queue._next_patch, expected_stdout=expected_stdout, expected_stderr=expected_stderr)

Ditto.

[Eric Seidel (no email)]

Created attachment 98787 [details]
Patch for landing

[WebKit Review Bot]

Comment on attachment 98787 [details]
Patch for landing

Rejecting attachment 98787 [details] from commit-queue.

Failed to run "['./Tools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=ec2-cq-01', '--port..." exit_code: 1

Last 500 characters of output:
139fd5220d9bbbf17b9ffc07a24c553015a8c25f
r89863 = 97c078b725714bdefe986a4ed07e31d1f9e900ab
Done rebuilding .git/svn/refs/remotes/origin/master/.rev_map.268f45cc-cd09-0410-ab3c-d52691b4dbfc
First, rewinding head to replay your work on top of it...
Fast-forwarded master to refs/remotes/origin/master.
Updating chromium port dependencies using gclient...

________ running '/usr/bin/python gyp_webkit' in '/mnt/git/webkit-commit-queue/Source/WebKit/chromium'
Updating webkit projects from gyp files...

Full output: http://queues.webkit.org/results/8948426

[WebKit Review Bot]

Comment on attachment 98787 [details]
Patch for landing

Clearing flags on attachment: 98787

Committed r89868: <http://trac.webkit.org/changeset/89868>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.

[Eric Seidel (no email)]

Created attachment 98801 [details]
Patch for landing

[Eric Seidel (no email)]

webkit-patch should auto-open bugs when uploading new patches. :(

[WebKit Review Bot]

Comment on attachment 98801 [details]
Patch for landing

Rejecting attachment 98801 [details] from commit-queue.

Failed to run "['./Tools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=ec2-cq-03', '--port..." exit_code: 2

Last 500 characters of output:
.rej
patching file Tools/Scripts/webkitpy/tool/commands/commandtest.py
Hunk #1 FAILED at 32.
1 out of 1 hunk FAILED -- saving rejects to file Tools/Scripts/webkitpy/tool/commands/commandtest.py.rej
patching file Tools/Scripts/webkitpy/tool/commands/queues_unittest.py
Hunk #1 FAILED at 166.
1 out of 1 hunk FAILED -- saving rejects to file Tools/Scripts/webkitpy/tool/commands/queues_unittest.py.rej

Failed to run "[u'/mnt/git/webkit-commit-queue/Tools/Scripts/svn-apply', u'--force']" exit_code: 1

Full output: http://queues.webkit.org/results/8954078

[Eric Seidel (no email)]

Was already landed.

[Ryosuke Niwa]

A python test is failing on Chromium Windows bots after this patch was landed:
http://build.webkit.org/builders/Chromium%20Win%20Release%20%28Tests%29/builds/16070/steps/webkitpy-test/logs/stdio

Traceback (most recent call last):
  File "E:\google-windows-2\chromium-win-release-tests\build\Tools\Scripts\webkitpy\layout_tests\port\webkit_unittest.py", line 124, in test_build_driver
    self.assertTrue(output.assert_outputs(self, port._build_driver, expected_stderr=expected_stderr))
  File "E:\google-windows-2\chromium-win-release-tests\build\Tools\Scripts\webkitpy\common\system\outputcapture.py", line 66, in assert_outputs
    testcase.assertEqual(stderr_string, expected_stderr)
AssertionError: "MOCK run_command: ['Tools\\\\Scripts\\\\build-dumprendertree', '--release']\n" != "MOCK run_command: ['Tools/Scripts/build-dumprendertree', '--release']\n"

[Eric Seidel (no email)]

OK.  Easy fix, will write it now.

[Eric Seidel (no email)]

reopen to fix windows.

[Eric Seidel (no email)]

Created attachment 98944 [details]
Patch

[Eric Seidel (no email)]

re-open to fix windows.

[Eric Seidel (no email)]

(In reply to comment #20)
> re-open to fix windows.

I finally fixed webkit-patch to know how to re-open bugs when posting new patches! bug 32006.

[WebKit Review Bot]

Comment on attachment 98944 [details]
Patch

Clearing flags on attachment: 98944

Committed r89962: <http://trac.webkit.org/changeset/89962>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.