Bug 175590 – run-webkit-test: ValueError: invalid literal for int() with base 10: 'null'

• Status: NEW
• Component: Tools / Tests

[Daniel Bates]

Created attachment 318143 [details]
Test

Perform the following:

1. Download and apply the attached patch.
2. From the top-level WebKit checkout directory, run `Tools/Scripts/run-webkit-tests -1 --debug --no-retry-failures --no-sample-on-timeout LayoutTests/http/tests/xmlhttprequest/testcase.html`.

Then run-webkit-tests dies with the following error:

ValueError raised: invalid literal for int() with base 10: 'null'

and a traceback of the form:

Traceback (most recent call last):
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 84, in main
    run_details = run(port, options, args, stderr)
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 443, in run
    run_details = manager.run(args)
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 249, in run
    initial_results, retry_results, enabled_pixel_tests_in_retry = self._run_test_subset(default_device_tests, tests_to_skip)
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 273, in _run_test_subset
    initial_results = self._run_tests(tests_to_run, tests_to_skip, self._options.repeat_each, self._options.iterations, int(self._options.child_processes), retrying=False)
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 330, in _run_tests
    return self._runner.run_tests(self._expectations, test_inputs, tests_to_skip, num_workers, retrying)
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 123, in run_tests
    pool.run(('test_list', shard.name, shard.test_inputs) for shard in all_shards)
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/common/message_pool.py", line 97, in run
    self.wait()
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/common/message_pool.py", line 127, in wait
    self._workers[0].run()
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/common/message_pool.py", line 267, in run
    self._raise(sys.exc_info())
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/common/message_pool.py", line 255, in run
    worker.handle(message.name, message.src, *message.args)
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 258, in handle
    self._run_test(test_input, test_list_name)
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 282, in _run_test
    result = self._run_test_with_or_without_timeout(test_input, test_timeout_sec, stop_when_done)
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 325, in _run_test_with_or_without_timeout
    return self._run_test_in_this_thread(test_input, stop_when_done)
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 406, in _run_test_in_this_thread
    return self._run_single_test(self._driver, test_input, stop_when_done)
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 410, in _run_single_test
    self._name, driver, test_input, stop_when_done)
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py", line 46, in run_single_test
    return runner.run()
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py", line 105, in run
    return self._run_compare_test()
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py", line 108, in _run_compare_test
    driver_output = self._driver.run_test(self._driver_input(), self._stop_when_done)
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/port/driver.py", line 692, in run_test
    return self._driver.run_test(driver_input, stop_when_done)
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/port/driver.py", line 206, in run_test
    text, audio = self._read_first_block(deadline, driver_input.test_name)  # First block is either text or audio
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/port/driver.py", line 512, in _read_first_block
    block = self._read_block(deadline, test_name)
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/port/driver.py", line 589, in _read_block
    self._process_stdout_line(block, out_line)
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/port/driver.py", line 541, in _process_stdout_line
    or self._read_header(block, line, 'Content-Length: ', '_content_length', int)
  File "/Volumes/Data/.../OpenSource/Tools/Scripts/webkitpy/port/driver.py", line 533, in _read_header
    value = header_filter(value)
ValueError: invalid literal for int() with base 10: 'null'

[Daniel Bates]

For some reason removing the following line from testcase.html does not cause run-webkit-test to die with an error:

log(`Content-Length: ${xhr.getResponseHeader('Content-Length2')}`);

[Daniel Bates]

run-webkit-tests seems to be interpreting the test output as HTTP headers.

[Jonathan Bedard]

What do you expect to happen here?  Obviously the exception is bad, but should this test pass?  Should we fail with a text-diff?  Should we restart the test runner?

[Daniel Bates]

Reduced test case:

<html>
<body>
<pre id="console"></pre>
<script>
if (window.testRunner)
    testRunner.dumpAsText();
document.getElementById('console').textContent = 'Content-Length: null';
</script>
</body>
</html>

[Daniel Bates]

(In reply to Jonathan Bedard from comment #3)
> What do you expect to happen here?  Obviously the exception is bad, but
> should this test pass?  Should we fail with a text-diff?  Should we restart
> the test runner?

We expect the test to run successfully and its output should be identical to the output if you ran the test in Safari.

[Daniel Bates]

Created attachment 318146 [details]
Reduced test case

[Daniel Bates]

Created attachment 318147 [details]
Reduced test case

[Daniel Bates]

*** Bug 175589 has been marked as a duplicate of this bug. ***