I have been checking the WebDriver tests from WPT on Epiphany and there are several tests that fail or timeout on Epiphany, but work with Minibrowser. With MiniBrowser: $ ./wpt run --webkit-port gtk --binary=/usr/lib/x86_64-linux-gnu/webkit2gtk-4.0/MiniBrowser --binary-arg=--automation \ --webdriver-binary=/usr/bin/WebKitWebDriver --webdriver-arg=--host=127.0.0.1 \ --log-html=results.html --log-mach=results.txt --log-wptreport=results.json --processes=8 \ webkit /webdriver [...] Ran 151 tests finished in 678.6 seconds. • 90 ran as expected. 0 tests skipped. • 11 tests timed out unexpectedly • 50 tests had unexpected subtest results With Epiphany: $ ./wpt run --webdriver-arg=--host=127.0.0.1 \ --log-html=results.html --log-mach=results.txt --log-wptreport=results.json --processes=8 \ epiphany /webdriver [...] Ran 151 tests finished in 680.2 seconds. • 73 ran as expected. 0 tests skipped. • 15 tests timed out unexpectedly • 63 tests had unexpected subtest results
Created attachment 379241 [details] simplified test case to reproduce on of the issues To test this test case do the following: 1) On one terminal run: WebKitWebDriver --host=127.0.0.1 --port=4444 2) On another terminal execute this script like follows: For MiniBrowser test: ./simplified_test_case_input_text.sh minibrowser /path/to/MiniBrowser For Epihany test: ./simplified_test_case_input_text.sh epiphany /path/to/epiphany (Restart the WebKitWebDriver on terminal 1) between each test). The test passes if you see how the text "foo" is wrote in the input text field of the page that opens (in the browser).
I have tried to run run-webdriver-tests with epiphany by modifying locally webdriver_driver_gtk.py, but it is not working at all. The epiphany window closes as soon as it opens and all the tests give an error. Not sure what is wrong with it.
(In reply to Carlos Alberto Lopez Perez from comment #2) > I have tried to run run-webdriver-tests with epiphany by modifying locally > webdriver_driver_gtk.py, but it is not working at all. > The epiphany window closes as soon as it opens and all the tests give an > error. > Not sure what is wrong with it. This was due to to the test-runner trying to use the wrong browser name in the capabilities negotiation when the webdriver was starting. I have a patch that allows to run the webkit webdriver tests with epiphany: https://people.igalia.com/clopez/wkbug/202047/support-epiphany-webdriver-tests.patch The instructions about how to use it, are in the patch itself (in the commit log). With it I get the following results: MiniBrowser (r250519): https://people.igalia.com/clopez/wkbug/202047/minibrowser_results.txt Unexpected failures (61) Epiphany (3.34 + WebKit r250519): https://people.igalia.com/clopez/wkbug/202047/epiphany_results.txt Unexpected failures (84) Unexpected timeouts (10)
The problem, at least one of them if there's more than one, is that epiphany keeps the focus on the location entry. Since the load is not triggered by ephy UI the focus is not transferred to the web view. I'll write a patch for ephy.
https://gitlab.gnome.org/GNOME/epiphany/commit/cd77e3944e4ebb6789ae59153e0f66994613719f I pushed it to 3.34 branch too.
Timeouts are gone, but I'm still getting more failures with ephy. I'm investigating. Some of them are related to the set/get window rect, there seems to be a race because they are flaky failures.
When it fails: webkitpy.webdriver_tests.webdriver_w3c_executor: [DEBUG] (WebKitWebDriver:16220): DBG: POST /session/976322be-cd32-448e-9be5-37b03334cb78/window/rect {"width": 800, "height": 600} webkitpy.webdriver_tests.webdriver_w3c_executor: [DEBUG] (WebKitWebDriver:16220): DBG: {"value":{"x":26,"y":23,"width":1024,"height":768}} webkitpy.webdriver_tests.webdriver_w3c_executor: [DEBUG] (WebKitWebDriver:16220): DBG: GET /session/976322be-cd32-448e-9be5-37b03334cb78/window/rect webkitpy.webdriver_tests.webdriver_w3c_executor: [DEBUG] (WebKitWebDriver:16220): DBG: {"value":{"x":26,"y":23,"width":1024,"height":768}} When it works: webkitpy.webdriver_tests.webdriver_w3c_executor: [DEBUG] (WebKitWebDriver:16139): DBG: POST /session/c6e754a5-7ac7-490c-8b63-ae1f7df438cf/window/rect {"width": 800, "height": 600} webkitpy.webdriver_tests.webdriver_w3c_executor: [DEBUG] (WebKitWebDriver:16139): DBG: {"value":{"x":26,"y":23,"width":800,"height":600}} webkitpy.webdriver_tests.webdriver_w3c_executor: [DEBUG] (WebKitWebDriver:16139): DBG: GET /session/c6e754a5-7ac7-490c-8b63-ae1f7df438cf/window/rect webkitpy.webdriver_tests.webdriver_w3c_executor: [DEBUG] (WebKitWebDriver:16139): DBG: {"value":{"x":26,"y":23,"width":800,"height":600}} I think we assume that setting the window size is sync while it's not. Sometimes it takes a bit more and we get the previous size. We don't notice it in MB because 800x600 is already the default window size of MB.