Windows regression: webkitpy.tool.steps.steps_unittest.StepsTest.test_runtests_jsc[_debug]

[Srinivasan Vijayaraghavan]

https://build.webkit.org/builders/Apple%20Win%207%20Release%20(Tests)/builds/62036/steps/webkitpy-test/logs/stdio

[1482/1532] webkitpy.tool.steps.steps_unittest.StepsTest.test_runtests_jsc failed:
  Traceback (most recent call last):
    File "/home/buildbot/slave/win-release-tests/build/Tools/Scripts/webkitpy/tool/steps/steps_unittest.py", line 160, in test_runtests_jsc
      OutputCapture().assert_outputs(self, step.run, [{}], expected_logs=expected_logs)
    File "/home/buildbot/slave/win-release-tests/build/Tools/Scripts/webkitpy/common/system/outputcapture.py", line 105, in assert_outputs
      testassert(logs_string, expected_logs)
  AssertionError: "MOCK run_command: ['perl', 'Tools/Scripts/webkit-build-directory', '--configura [truncated]... != "MOCK run_command: ['perl', 'Tools/Scripts/webkit-build-directory', '--configura [truncated]...
    MOCK run_command: ['perl', 'Tools/Scripts/webkit-build-directory', '--configuration', '--release', '--mac'], cwd=/mock-checkout
  - MOCK run_and_throw_if_fail: ['Tools/Scripts/run-javascriptcore-tests', '--no-fail-fast', '--release', '--json-output=MOCK output of child process/jsc_test_results.json'], cwd=/mock-checkout
  + MOCK run_and_throw_if_fail: ['Tools/Scripts/run-javascriptcore-tests', '--no-fail-fast', '--release', '--json-output=/MOCK output of child process/jsc_test_results.json'], cwd=/mock-checkout
  ?                                                                                                                      +
  
  
[1482/1532] webkitpy.tool.steps.steps_unittest.StepsTest.test_runtests_jsc_debug
[1483/1532] webkitpy.tool.steps.steps_unittest.StepsTest.test_runtests_jsc_debug failed:
  Traceback (most recent call last):
    File "/home/buildbot/slave/win-release-tests/build/Tools/Scripts/webkitpy/tool/steps/steps_unittest.py", line 174, in test_runtests_jsc_debug
      OutputCapture().assert_outputs(self, step.run, [{}], expected_logs=expected_logs)
    File "/home/buildbot/slave/win-release-tests/build/Tools/Scripts/webkitpy/common/system/outputcapture.py", line 105, in assert_outputs
      testassert(logs_string, expected_logs)
  AssertionError: "MOCK run_command: ['perl', 'Tools/Scripts/webkit-build-directory', '--configura [truncated]... != "MOCK run_command: ['perl', 'Tools/Scripts/webkit-build-directory', '--configura [truncated]...
    MOCK run_command: ['perl', 'Tools/Scripts/webkit-build-directory', '--configuration', '--release', '--mac'], cwd=/mock-checkout
  - MOCK run_and_throw_if_fail: ['Tools/Scripts/run-javascriptcore-tests', '--no-fail-fast', '--debug', '--json-output=MOCK output of child process/jsc_test_results.json'], cwd=/mock-checkout
  + MOCK run_and_throw_if_fail: ['Tools/Scripts/run-javascriptcore-tests', '--no-fail-fast', '--debug', '--json-output=/MOCK output of child process/jsc_test_results.json'], cwd=/mock-checkout
  ?                                                                                                                    +

  
Mock output does not include a trailing slash on Windows, which leads to test failures on Windows bots.

[Srinivasan Vijayaraghavan]

Created attachment 302573 [details]
Patch

[Build Bot]

Comment on attachment 302573 [details]
Patch

Attachment 302573 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/3181306

New failing tests:
media/modern-media-controls/volume-down-support/volume-down-support.html

[Build Bot]

Created attachment 302586 [details]
Archive of layout-test-results from ews112 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews112  Port: mac-elcapitan  Platform: Mac OS X 10.11.6

[Per Arne Vollan]

Comment on attachment 302573 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=302573&action=review

> Tools/Scripts/webkitpy/tool/steps/steps_unittest.py:153
> +

Perhaps you could add a method jsc_results_file_path() in runtests.py (see _run_javascriptcore_tests()), instead of hardcoding the string here?

> Tools/Scripts/webkitpy/tool/steps/steps_unittest.py:166
> +""" % (self._jsc_output_dir())

Use the jsc_results_file_path() function here.

> Tools/Scripts/webkitpy/tool/steps/steps_unittest.py:180
> +""" % (self._jsc_output_dir())

Ditto.

[Srinivasan Vijayaraghavan]

(In reply to comment #4)
> Comment on attachment 302573 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=302573&action=review
> 
> > Tools/Scripts/webkitpy/tool/steps/steps_unittest.py:153
> > +
> 
> Perhaps you could add a method jsc_results_file_path() in runtests.py (see
> _run_javascriptcore_tests()), instead of hardcoding the string here?
> 
I'm not entirely sure I understand. Wouldn't that remove test coverage for the output path?

In other words, won't the --json-output= part match even if bad changes are made elsewhere (not just in runtests, but also in port, filesystem etc)? Hard-coding the expected unit test output would guard against this.

[Alexey Proskuryakov]

Comment on attachment 302573 [details]
Patch

Sri is going to avoid running the webkit-build-directory script, so we won't have "MOCK output of child process" in the path.

[Srinivasan Vijayaraghavan]

Created attachment 302899 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 302899 [details]
Patch

Clearing flags on attachment: 302899

Committed r213114: <http://trac.webkit.org/changeset/213114>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Per Arne Vollan]

(In reply to comment #7)
> Created attachment 302899 [details]
> Patch

Thanks for fixing this!