Bug 242763 – run-webkit-tests -p svg does not finish (Python assertion)

• Component: SVG
• Status: RESOLVED FIXED

[Nikolas Zimmermann]

Simply invoke "run-webkit-tests -p svg" on macOS Monterey on a vanilla WebKit main branch build:

...
worker/1 stopping
worker/4 stopping                                
[1206/2814] svg/filters/big-sized-filter.svg failed unexpectedly (image diff)
worker/3 stopping

AttributeError raised: 'tuple' object has no attribute 'passed' (from worker/5)
Traceback (most recent call last):
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 92, in main
    run_details = run(port, options, args, stderr)
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 495, in run
    run_details = manager.run(args)
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 413, in run
    temp_initial_results, temp_retry_results, temp_enabled_pixel_tests_in_retry = self._run_test_subset(test_inputs, device_type=device_type)
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 487, in _run_test_subset
    initial_results = self._run_tests(test_inputs, self._options.repeat_each, self._options.iterations, int(self._options.child_processes), retrying=False, device_type=device_type)
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 566, in _run_tests
    return self._runner.run_tests(self._expectations[device_type], new_test_inputs, num_workers, retrying, device_type)
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 179, in run_tests
    pool.wait()
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/task_pool.py", line 426, in wait
    self.queue.receive()(self)
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/task_pool.py", line 133, in __call__
    reraise(*self.exc_info)
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/libraries/autoinstalled/python-3-arm64/six.py", line 702, in reraise
    raise value.with_traceback(tb)
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/task_pool.py", line 299, in main
    queue.send(_Result(value=task(None), id=task.id))
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/task_pool.py", line 56, in __call__
    return self.function(*self.args, **self.kwargs)
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 76, in run_shard
    return Worker.instance.run_tests(shard)
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 318, in run_tests
    Worker.instance.run_test(input, shard.name)
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 346, in run_test
    result = self._run_test_with_or_without_timeout(test_input, test_timeout_sec, stop_when_done)
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 403, in _run_test_with_or_without_timeout
    return self._run_test_in_this_thread(test_input, stop_when_done)
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 489, in _run_test_in_this_thread
    return self._run_single_test(self._driver, test_input, stop_when_done)
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 492, in _run_single_test
    return single_test_runner.run_single_test(
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py", line 48, in run_single_test
    return runner.run()
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py", line 128, in run
    return self._run_compare_test()
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py", line 146, in _run_compare_test
    test_result = self._compare_output(expected_driver_output, driver_output)
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py", line 282, in _compare_output
    failures.extend(self._compare_image(expected_driver_output, driver_output))
  File "/Users/nzimmermann/Software/GitRepositories/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py", line 330, in _compare_image
    if not diff_result.passed:
AttributeError: 'tuple' object has no attribute 'passed' (from worker/5)

Quite annoying, need to investigate.

[Nikolas Zimmermann]

ImageDiff apparently didn't execute successfully:

This is the "diff_result" object, where no "passed" or "error_string" attribute is present, in the case of a ImageDiff problem:
(None, 0, 'Failed to compute an image diff: [Errno 32] Broken pipe')

Here's a single test reproducer: run-webkit-tests -p svg/foreignObject/body-background.svg

[Nikolas Zimmermann]

Aha, here's the culprit:

nzimmermann@nzimmermann-macbook WebKit % DYLD_FRAMEWORK_PATH=$PWD/WebKitBuild/Debug/ $PWD/WebKitBuild/Debug/ImageDiff actual.png LayoutTests/svg/foreignObject/body-background-expected.png
Error: test and reference images differ in alpha. Test image has alpha, reference image does not have alpha.
nzimmermann@nzimmermann-macbook WebKit % echo $?
1


actual.png: PNG image data, 800 x 600, 8-bit/color RGBA, non-interlaced
nzimmermann@nzimmermann-macbook WebKit % file LayoutTests/svg/foreignObject/body-background-expected.png
LayoutTests/svg/foreignObject/body-background-expected.png: PNG image data, 800 x 600, 8-bit/color RGB, non-interlaced

This is a truly outdated expected.png, dating back to 2012, added by Florin.
I couldn't even remember we generated RGB images in the past...


Anyhow, I'm aware people rarely use pixel tests... will fix this on my own.

[Nikolas Zimmermann]

Pull request: https://github.com/WebKit/WebKit/pull/2446

[EWS]

Committed 252498@main (538f5387b7f6): <https://commits.webkit.org/252498@main>

Reviewed commits have been landed. Closing PR #2446 and removing active labels.

[Radar WebKit Bug Importer]

<rdar://problem/97068989>