Bug 169777 - [ios-simulator] webkitpy.tool.steps.steps_unittest.StepsTest.test_runtests_bindings is a flaky failure
Summary: [ios-simulator] webkitpy.tool.steps.steps_unittest.StepsTest.test_runtests_bi...
Status: RESOLVED DUPLICATE of bug 177751
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
Keywords: InRadar
Depends on:
Reported: 2017-03-16 12:54 PDT by Ryan Haddad
Modified: 2017-10-02 10:49 PDT (History)
5 users (show)

See Also:


Note You need to log in before you can comment on or make changes to this bug.
Description Ryan Haddad 2017-03-16 12:54:27 PDT
[1015/1548] webkitpy.tool.steps.steps_unittest.StepsTest.test_runtests_bindings failed:
  Traceback (most recent call last):
    File "/Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/Tools/Scripts/webkitpy/tool/steps/steps_unittest.py", line 258, in test_runtests_bindings
      OutputCapture().assert_outputs(self, step.run, [{}], expected_logs=expected_logs)
    File "/Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/Tools/Scripts/webkitpy/common/system/outputcapture.py", line 105, in assert_outputs
      testassert(logs_string, expected_logs)
  AssertionError: "MOCK run_command: ['perl', 'Tools/Scripts/webkit-build-directory', '--mac'], cw [truncated]... != "MOCK run_and_throw_if_fail: ['Tools/Scripts/run-bindings-tests', '--json-output [truncated]...
  - MOCK run_command: ['perl', 'Tools/Scripts/webkit-build-directory', '--mac'], cwd=/mock-checkout
    MOCK run_and_throw_if_fail: ['Tools/Scripts/run-bindings-tests', '--json-output=/tmp/bindings_test_results.json'], cwd=/mock-checkout

Comment 1 Ryan Haddad 2017-03-16 12:54:46 PDT
Comment 2 Srinivasan Vijayaraghavan 2017-03-16 17:32:57 PDT
build_directory() in webkitpy/port/config.py runs the webkit-build-directory command.

build_path() in webkitpy/port/base.py sometimes calls the above function if it doesn't have it cached. AIUI the order in which the tests are run using webkitpy is non-deterministic, so that might be the source of flakiness here.

If that is the case, the best solution would be to just make the webkit-build-directory line optional for the unit tests somehow.
Comment 3 Srinivasan Vijayaraghavan 2017-03-17 15:17:08 PDT
Adding some color to my previous comment: Running webkit-build-directory is done as a reaction to a cache miss inside webkitpy, so it's not strictly an error.
Comment 4 Ryan Haddad 2017-10-02 10:49:12 PDT

*** This bug has been marked as a duplicate of bug 177751 ***