I tried run nrwt but it crashed with the -p switch. Crash log the following: worker/0 raised TypeError('must be string or buffer, not bool'): layout_tests/controllers/worker.py:91 (in run) self._worker_connection.run_message_loop() layout_tests/controllers/message_broker.py:191 (in run_message_loop) self._broker.run_message_loop(self._run_topic, self._client, delay_secs) layout_tests/controllers/message_broker.py:127 (in run_message_loop) self._run_loop(topic_name, client, block=True, delay_secs=delay_secs) layout_tests/controllers/message_broker.py:141 (in _run_loop) self._dispatch_message(msg, client) layout_tests/controllers/message_broker.py:150 (in _dispatch_message) message_handler(message.src, *optargs) layout_tests/controllers/worker.py:111 (in handle_test_list) self._run_test(test_input) layout_tests/controllers/worker.py:126 (in _run_test) result = self.run_test_with_timeout(test_input, test_timeout_sec) layout_tests/controllers/worker.py:165 (in run_test_with_timeout) return self._run_test_in_this_thread(test_input) layout_tests/controllers/worker.py:250 (in _run_test_in_this_thread) return self.run_single_test(self._driver, test_input) layout_tests/controllers/worker.py:254 (in run_single_test) test_input, driver, self._name) layout_tests/controllers/single_test_runner.py:46 (in run_single_test) return runner.run() layout_tests/controllers/single_test_runner.py:121 (in run) return self._run_reftest() layout_tests/controllers/single_test_runner.py:290 (in _run_reftest) test_result_writer.write_test_result(self._port, self._test_name, driver_output1, driver_output2, test_result.failures) layout_tests/layout_package/test_result_writer.py:75 (in write_test_result) writer.write_image_diff_files(image_diff) layout_tests/layout_package/test_result_writer.py:208 (in write_image_diff_files) fs.write_binary_file(diff_filename, image_diff) common/system/filesystem.py:205 (in write_binary_file) f.write(contents) In my opinion with the --tolerance switch doesn't run the pixel tests on WK2.
General specification: CPU: Intel(R) Core(TM) i5-2320 CPU @ 3.00GHz RAM: 4 GB GPU: GeForce GT 440, 1 GB memory OS: Linux Ubuntu ----- My WebKit revision is 99400 (hash: 09a4c196f864ce8ca268cc1565cb408a55af0045) WK2 is built with Qt5. My Qt 5 hash code: 90f7cf110710bb59c2993d6d1848223baeb9642b ----- Experiences: When I want to run the WK2 pixel test [command: run-webkit-tests -2 -p], it starts at 43% and it runs to 60%, when it crashes with the wrote errors. When I use --tolerace switch [ command: run-webkit-tests -2 -p --tolerance 1 ], it doesn't run the pixel tests on WK2, but with this switch, it crashes as well... (it breaks at 60% as well...)
(In reply to comment #1) > When I want to run the WK2 pixel test [command: run-webkit-tests -2 -p], it starts at 43% and it runs to 60%, when it crashes with the wrote errors. > Try using the --verbose switch, which might give you more information (than the percentage) on which tests crash.
This test caused the crash: editing/pasteboard/data-transfer-items-image-png.html If I skipped that, the pixel test works!
(In reply to comment #3) > This test caused the crash: editing/pasteboard/data-transfer-items-image-png.html > If I skipped that, the pixel test works! I can confirm that behaviour! We should skip this test until a NRWT expert fix this issue!
Created attachment 114226 [details] skipped one test to solve the problem
Comment on attachment 114226 [details] skipped one test to solve the problem View in context: https://bugs.webkit.org/attachment.cgi?id=114226&action=review > LayoutTests/platform/qt-5.0/Skipped:1556 > +editing/pasteboard/data-transfer-items-image-png.html Would be good to have a comment for this entry refering to the bug URL, so we do not lose this information between all the skipped editing tests.
Comment on attachment 114226 [details] skipped one test to solve the problem View in context: https://bugs.webkit.org/attachment.cgi?id=114226&action=review > LayoutTests/ChangeLog:8 > + Reviewed by NOBODY (OOPS!). > + > + * platform/qt-5.0/Skipped: add to skipped: editing/pasteboard/data-transfer-items-image-png.html You should mention the reason in the ChangeLog file why you are putting this test to the Skipped list.
Created attachment 114229 [details] it solves the nrwt pixel test crash
Created attachment 114244 [details] nrwt pixeltest crash in WK2 resolved
Comment on attachment 114244 [details] nrwt pixeltest crash in WK2 resolved r=me
Comment on attachment 114244 [details] nrwt pixeltest crash in WK2 resolved The skip patch is landed: http://trac.webkit.org/changeset/99687
Reopen, because the test is still in skipped list.
Can someone confirm whether or not there is an actual problem in NRWT and we still get python crashes?
I'll take a look.
No longer crashing. The test editing/pasteboard/data-transfer-items-image-png.html still fails on qt, but for a different reason (different results). I'm closing this bug and reporting another one to keep track of that failure.