Bug 108353 – REGRESSION(r141205): Command for running tests on windows needs to be investigated. (Requested by lforschler on #webkit).

• Component: New Bugs
• Status: RESOLVED FIXED

[WebKit Review Bot]

http://trac.webkit.org/changeset/141205 broke the build:
Command for running tests on windows needs to be investigated. (Requested by lforschler on #webkit).

This is an automatic bug report generated by the sheriff-bot. If this bug
report was created because of a flaky test, please file a bug for the flaky
test (if we don't already have one on file) and dup this bug against that bug
so that we can track how often these flaky tests case pain.

"Only you can prevent forest fires." -- Smokey the Bear

[WebKit Review Bot]

Created attachment 185516 [details]
ROLLOUT of r141205

Any committer can land this patch automatically by marking it commit-queue+.  The commit-queue will build and test the patch before landing to ensure that the rollout will be successful.  This process takes approximately 15 minutes.

If you would like to land the rollout faster, you can use the following command:

  webkit-patch land-attachment ATTACHMENT_ID

where ATTACHMENT_ID is the ID of this attachment.

[Lucas Forschler]

I'm rolling this out.  It looks like there is a problem with the command being used to kick off EWS tests on windows.  Here is the output.

Failed to run "['/home/buildbot/WebKit/Tools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=APPLE-EWS-1', 'build-and-test', '--no-clean', '--no-update', '--test', '--non-interactive', '--port=win']" exit_code: 2 cwd: /home/buildbot/WebKit

Running run-webkit-tests
Failed to run "['Tools/Scripts/run-webkit-tests', '--no-new-test-results', '--no-show-results', '--skip-failing-tests', '--exit-after-n-failures=30', '--quiet']" exit_code: 1

Last 500 characters of output:
only)
  -v|--verbose                    More verbose output (overrides --quiet)
  -m|--merge-leak-depth arg       Merges leak callStacks and prints the number of unique leaks beneath a callstack depth of arg.  Defaults to 5.
  --use-remote-links-to-tests     Link to test files within the SVN repository in the results.
  -2|--webkit-test-runner         Use WebKitTestRunner rather than DumpRenderTree.
Failed to execute Tools/Scripts/old-run-webkit-tests at Tools/Scripts/run-webkit-tests line 106.

Failed to run "['Tools/Scripts/run-webkit-tests', '--no-new-test-results', '--no-show-results', '--skip-failing-tests', '--exit-after-n-failures=30', '--quiet']" exit_code: 1
Unknown option: skip-failing-tests
Usage: old-run-webkit-tests [options] [testdir|testpath ...]
  --add-platform-exceptions       Put new results for non-platform-specific failing tests into the platform-specific results directory
  --additional-platform-directory path/to/directory
                                  Look in the specified directory before looking in any of the default platform-specific directories
  --complex-text                  Use the complex text code path for all text (Mac OS X and Windows only)
  -c|--configuration config       Set DumpRenderTree build configuration
  --gc-between-tests              Force garbage collection between each test
  -g|--guard-malloc               Enable Guard Malloc
  --exit-after-n-failures N       Exit after the first N failures (includes crashes) instead of running all tests
  --exit-after-n-crashes-or-timeouts N
                                  Exit after the first N crashes instead of running all tests
  -h|--help                       Show this help message
  --[no-]http                     Run (or do not run) http tests (default: run)
  --[no-]wait-for-httpd           Wait for httpd if some other test session is using it already (same as WEBKIT_WAIT_FOR_HTTPD=1). (default: 0) 
  -i|--ignore-tests               Comma-separated list of directories or tests to ignore
  --iterations n                  Number of times to run the set of tests (e.g. ABCABCABC)
  --[no-]launch-safari            Launch (or do not launch) Safari to display test results (default: launch)
  --[no-]show-results             Same as --[no-]launch-safari
  -l|--leaks                      Enable leaks checking
  --[no-]new-test-results         Generate results for new tests
  --nthly n                       Restart DumpRenderTree every n tests (default: 1000)
  -p|--pixel-tests                Enable pixel tests
  --tolerance t                   Ignore image differences less than this percentage (default: 0)
  --platform                      Override the detected platform to use for tests and results (default: win)
  --port                          Web server port to use with http tests
  -q|--quiet                      Less verbose output
  --reset-results                 Reset ALL results (including pixel tests if --pixel-tests is set)
  -o|--results-directory          Output results directory (default: /tmp/layout-test-results)
  --random                        Run the tests in a random order
  --repeat-each n                 Number of times to run each test (e.g. AAABBBCCC)
  --reverse                       Run the tests in reverse alphabetical order
  --root                          Path to root tools build
  --[no-]sample-on-timeout        Run sample on timeout (default: run) (Mac OS X only)
  -1|--singly                     Isolate each test case run (implies --nthly 1 --verbose)
  --skipped=[default|ignore|only] Specifies how to treat the Skipped file
                                     default: Tests/directories listed in the Skipped file are not tested
                                     ignore:  The Skipped file is ignored
                                     only:    Only those tests/directories listed in the Skipped file will be run
  --slowest                       Report the 10 slowest tests
  --ignore-metrics                Ignore metrics in tests
  --[no-]strip-editing-callbacks  Remove editing callbacks from expected results
  -t|--threaded                   Run a concurrent JavaScript thead with each test
  --timeout t                     Sets the number of seconds before a test times out (default: 35)
  --valgrind                      Run DumpRenderTree inside valgrind (Qt/Linux only)
  -v|--verbose                    More verbose output (overrides --quiet)
  -m|--merge-leak-depth arg       Merges leak callStacks and prints the number of unique leaks beneath a callstack depth of arg.  Defaults to 5.
  --use-remote-links-to-tests     Link to test files within the SVN repository in the results.
  -2|--webkit-test-runner         Use WebKitTestRunner rather than DumpRenderTree.
Failed to execute Tools/Scripts/old-run-webkit-tests at Tools/Scripts/run-webkit-tests line 106.

[WebKit Review Bot]

Comment on attachment 185516 [details]
ROLLOUT of r141205

Clearing flags on attachment: 185516

Committed r141337: <http://trac.webkit.org/changeset/141337>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.

[Roger Fong]

Reopening for resolution

[Roger Fong]

Created attachment 185623 [details]
patch to fix EWS

--skip-failing-tests is not an option for old-run-webkit-tests (which is what is used by windows port).
Patch adds a platform check for cygwin (aka windows) to make sure that the option isn't added when test are run on windows.

[Roger Fong]

Comment on attachment 185623 [details]
patch to fix EWS

Rubberstamped by thorton.
Committed http://trac.webkit.org/changeset/141341