Bug 209897

Summary: [Windows] Ignore render tree dump based results on Windows
Product: WebKit Reporter: Antti Koivisto <koivisto>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: ap, bfulgham, darin, ews-watchlist, glenn, jbedard, simon.fraser, tsavell, webkit-bug-importer, zalan
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
patch
none
patch
none
patch none

Description Antti Koivisto 2020-04-02 01:51:54 PDT
Render tree dumps are port specific and so difficult to maintain. They are mostly very old and have been superseded by more modern ref and text dump tests. They rarely reveal any port specific bugs.
Comment 1 Antti Koivisto 2020-04-02 01:59:48 PDT
Created attachment 395251 [details]
patch
Comment 2 Antti Koivisto 2020-04-02 02:07:43 PDT
Created attachment 395252 [details]
patch
Comment 3 EWS 2020-04-03 00:08:47 PDT
Committed r259446: <https://trac.webkit.org/changeset/259446>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 395252 [details].
Comment 4 Radar WebKit Bug Importer 2020-04-03 00:09:17 PDT
<rdar://problem/61247373>
Comment 5 Truitt Savell 2020-04-07 13:23:56 PDT
Reverted r259446 for reason:

Broke Windows testing

Committed r259667: <https://trac.webkit.org/changeset/259667>
Comment 6 Ryan Haddad 2020-04-07 13:26:24 PDT
Example of failure:
https://build.webkit.org/builders/Apple%20Win%2010%20Release%20%28Tests%29/builds/5456

02:05:39.348 2864 worker/2: TypeError('expected string or buffer') raised:
02:05:39.352 2864     File "/home/buildbot/worker/win10-release-tests/build/Tools/Scripts/webkitpy/common/message_pool.py", line 280, in run
02:05:39.352 2864       worker.handle(message.name, message.src, *message.args)
02:05:39.352 2864     File "/home/buildbot/worker/win10-release-tests/build/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 291, in handle
02:05:39.352 2864       self._run_test(test_input, test_list_name)
02:05:39.352 2864     File "/home/buildbot/worker/win10-release-tests/build/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 317, in _run_test
02:05:39.352 2864       result = self._run_test_with_or_without_timeout(test_input, test_timeout_sec, stop_when_done)
02:05:39.352 2864     File "/home/buildbot/worker/win10-release-tests/build/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 383, in _run_test_with_or_without_timeout
02:05:39.352 2864       return self._run_test_in_this_thread(test_input, stop_when_done)
02:05:39.353 2864     File "/home/buildbot/worker/win10-release-tests/build/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 469, in _run_test_in_this_thread
02:05:39.353 2864       return self._run_single_test(self._driver, test_input, stop_when_done)
02:05:39.353 2864     File "/home/buildbot/worker/win10-release-tests/build/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 473, in _run_single_test
02:05:39.353 2864       self._name, driver, test_input, stop_when_done)
02:05:39.353 2864     File "/home/buildbot/worker/win10-release-tests/build/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py", line 47, in run_single_test
02:05:39.353 2864       return runner.run()
02:05:39.353 2864     File "/home/buildbot/worker/win10-release-tests/build/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py", line 106, in run
02:05:39.353 2864       return self._run_compare_test()
02:05:39.353 2864     File "/home/buildbot/worker/win10-release-tests/build/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py", line 122, in _run_compare_test
02:05:39.353 2864       test_result = self._compare_output(expected_driver_output, driver_output)
02:05:39.353 2864     File "/home/buildbot/worker/win10-release-tests/build/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py", line 230, in _compare_output
02:05:39.353 2864       failures.extend(self._compare_text(expected_driver_output.text, driver_output.text))
02:05:39.353 2864     File "/home/buildbot/worker/win10-release-tests/build/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py", line 238, in _compare_text
02:05:39.353 2864       if self._options.ignore_render_tree_dump_results and self._render_tree_dump_pattern.match(actual_text):
Comment 7 Truitt Savell 2020-04-07 14:27:37 PDT
After this rollout Windows testing is working again https://build.webkit.org/builders/Apple%20Win%2010%20Release%20%28Tests%29/builds/5546
Comment 8 Antti Koivisto 2020-09-23 05:28:45 PDT
Created attachment 409464 [details]
patch
Comment 9 Antti Koivisto 2020-09-23 05:29:28 PDT
The failures were because some tests produce null actual_text
Comment 10 EWS 2020-09-23 21:33:28 PDT
Committed r267517: <https://trac.webkit.org/changeset/267517>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 409464 [details].