Running iOS layout tests sometimes fails with a utf-8 error

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Simon Fraser (smfr)]

Running iOS layout tests sometimes fails with a script error:

run-webkit-tests --ios-simulator LayoutTests/imported/w3c/web-platform-tests/css/css-transforms

[This is with a local import of css-transforms WPT tests].

[116/790] imported/w3c/web-platform-tests/css/css-transforms/transform3d-backface-visibility-001.html passed                                  
UnicodeDecodeError raised: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte
Traceback (most recent call last):
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 92, in main
    run_details = run(port, options, args, stderr)
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 494, in run
    run_details = manager.run(args)
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 413, in run
    temp_initial_results, temp_retry_results, temp_enabled_pixel_tests_in_retry = self._run_test_subset(test_inputs, device_type=device_type)
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 487, in _run_test_subset
    initial_results = self._run_tests(test_inputs, self._options.repeat_each, self._options.iterations, int(self._options.child_processes), retrying=False, device_type=device_type)
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 566, in _run_tests
    return self._runner.run_tests(self._expectations[device_type], new_test_inputs, num_workers, retrying, device_type)
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 175, in run_tests
    pool.do(
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/task_pool.py", line 397, in do
    result = function(*args, **kwargs)
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 76, in run_shard
    return Worker.instance.run_tests(shard)
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 318, in run_tests
    Worker.instance.run_test(input, shard.name)
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 346, in run_test
    result = self._run_test_with_or_without_timeout(test_input, test_timeout_sec, stop_when_done)
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 403, in _run_test_with_or_without_timeout
    return self._run_test_in_this_thread(test_input, stop_when_done)
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 489, in _run_test_in_this_thread
    return self._run_single_test(self._driver, test_input, stop_when_done)
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 492, in _run_single_test
    return single_test_runner.run_single_test(
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py", line 46, in run_single_test
    return runner.run()
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py", line 118, in run
    return self._run_reftest()
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py", line 328, in _run_reftest
    reference_output = self._driver.run_test(DriverInput(reference_test_name, self._timeout, None, should_run_pixel_test=True), self._stop_when_done)
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/webkitpy/port/driver.py", line 834, in run_test
    return self._driver.run_test(driver_input, stop_when_done)
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/webkitpy/port/driver.py", line 246, in run_test
    text += string_utils.decode(out, target_type=str)
  File "/Volumes/Data/Development/system/webkit/OpenSource/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/string_utils.py", line 46, in decode
    return data.decode(encoding, errors=errors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte
Stopping Web Platform Test server ...%

Python 2.7.16

[Jonathan Bedard]

Is this a Python 2.7 error? run-webkit-tests should be totally Python 3 now, but I would also be surprised if we get this error in Python 2

[Simon Fraser (smfr)]

Not sure if it uses Python 2.7 when running; that's just what `python --version` returns for me.

Adding some logging suggests that this is raw PNG data:

<class 'UnicodeDecodeError'>
b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x06@\x00\x00\x04\xb0\x08\x06\x00\x00\x00\xa3\x01\x86\x97\x00\x00\x00)tEXtchecksum\x0015433a077b92dfc7130ccbf0d4b617de\\\xbc\xf1}\x00\x00\x00\x01sRGB\x00\xae\xce\x1c\xe9\x00\x00\x00DeXIfMM\x00*\x00\x00\x00\x08\x00\x01\x87i\x00\x04\x00\x00\x00\x01\x00\x00\x00\x1a\x00\x00\x00\x00\x00\x03\xa0\x01\x00\x03\x00\x00\x00\x01\x00\x01\x00\x00\xa0\x02\x00\x04\x00\x00\x00\x01\x00\x00\x06@\xa0\x03\x00\x04\x00\x00\x00\x01\x00\x00\x04\xb0\x00\x00\x00\x00\x9b\x88\xec~\x00\x00@\x00IDATx\x01\xec\xdd\t\xdc5U]8\xf0\x91\xcc\xcc\xa5$+B\x84 ,C\xd1p\xc1\x9d4\x88\x94\xa4\x12SA44\xf7\\\x12P\xc1\x05\xc4\x05+\xdc\x15wE\xc5"\x05s\t\x14\x94\xc0%\x15\\XT\x02\x95,"\x97"\xcd-5\x97r\xfe\xe7w\xfa\x9fq\xee}\xee}\x9e\xe7\xce\xbd\xf7}\x9e\xe7\xf0\x9d\xcf\xe7}\xef}f?\xdfsf\xee\xcc\xf9\x9d9s\x8d6\r\x8d\x81\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 P\x91\xc0v\x15\xa5ER\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02Y@\x00DA @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\xaa\x13\x10\x00\xa9.K%\x88\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10\x10\x00Q\x06\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x81\xea\x04\x04@\xaa\xcbR\t"@\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x04@\x94\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\xa0:\x01\x01\x90\xea\xb2T\x82\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x01\x10e\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10\xa8N@\x00\xa4\xba,\x95 \x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @@\x00D\x19 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\xaa\x13\x10\x00\xa9.K%\x88\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10\x10\x00Q\x06\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x81\xea\x04\x04@\xaa\xcbR\t"@\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x04@\x94\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\xa0:\x01\x01\x90\xea\xb2T\x82\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x01\x10e\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10\xa8N@\x00\xa4\xba,\x95 \x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @@\x00D\x19 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\xaa\x13\x10\x00\xa9.K%\x88\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10\x10\x00Q\x06\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x01\x02\x04\x08\x10 @\x80\x00\x81\xea\x04\x

Why are we trying to decode this as Unicode?

[Jonathan Bedard]

We seem to think that output should be text? Is an image defined there? Seems like a logic error in this function.

To clarify, your checkout has a python3 shebang in run-webkit-tests, right?

[Simon Fraser (smfr)]

#!/usr/bin/env python3

yes

[Simon Fraser (smfr)]

This happens for me when a test times out; at driver.py:264 we're just dumping the rest of stdout but here it contains PNG data, which is normally processed by _read_block.

[Simon Fraser (smfr)]

Workaround:

diff --git a/Tools/Scripts/webkitpy/port/driver.py b/Tools/Scripts/webkitpy/port/driver.py
index fca1bad07e53098bb03b2803018484949a66fc13..1314151776b770128cf0ae4c333dad04acf4f84d 100644
--- a/Tools/Scripts/webkitpy/port/driver.py
+++ b/Tools/Scripts/webkitpy/port/driver.py
@@ -237,13 +237,14 @@ class Driver(object):
         pid = self._server_process.pid()
 
         if stop_when_done or crashed or timed_out:
+            print("Stopped %s crashed %s timed_out %s\n" % (stop_when_done, crashed, timed_out))
             if stop_when_done and not (crashed or timed_out):
                 self.do_post_tests_work()
             # We call stop() even if we crashed or timed out in order to get any remaining stdout/stderr output.
             # In the timeout case, we kill the hung process as well.
             out, err = self._server_process.stop(self._port.driver_stop_timeout() if stop_when_done else 0.0)
-            if out:
-                text += string_utils.decode(out, target_type=str)
+            # if out:
+            #     text += string_utils.decode(out, target_type=str)
             if err:
                 self.error_from_test += string_utils.decode(err, target_type=str)
             self._server_process = None

[Jonathan Bedard]

Created attachment 437764 [details]
Patch

[Alexey Proskuryakov]

Comment on attachment 437764 [details]
Patch

Is this testable?

[Jonathan Bedard]

(In reply to Alexey Proskuryakov from comment #8)
> Comment on attachment 437764 [details]
> Patch
> 
> Is this testable?

Not easily.

The fix is in "run_test", and looking through our existing tests, we don't call that function outside of production code.

[EWS]

Committed r282347 (241613@main): <https://commits.webkit.org/241613@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 437764 [details].

[Radar WebKit Bug Importer]

<rdar://problem/83066386>