Bug 230551 – [WebDriver] Update w3c and selenium tests

• Status: RESOLVED
• Resolution: FIXED
• Component: WebDriver

[Carlos Garcia Campos]

It's been a while since the last time we imported w3c and selenium tests. They both now require python3 and a new version of pytest, so it requires changes in our test runner too.

[Carlos Garcia Campos]

Created attachment 438811 [details]
Patch

[Carlos Garcia Campos]

I still have to do the gardening, there are a lot of new failures...

In w3c:

Unexpected failures (186)
  imported/w3c/webdriver/tests/switch_to_parent_frame/switch.py::test_switch_from_iframe
  imported/w3c/webdriver/tests/find_elements_from_element/find.py::test_find_elements_link_text[<a href=#>link<br>text</a>-link\ntext]
  imported/w3c/webdriver/tests/find_elements_from_element/find.py::test_find_elements_partial_link_text[<a href=#>partial link<br>text</a>-k\nt]
  imported/w3c/webdriver/tests/is_element_enabled/enabled.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/find_element_from_element/find.py::test_find_element_link_text[<a href=#>link<br>text</a>-link\ntext]
  imported/w3c/webdriver/tests/find_element_from_element/find.py::test_find_element_partial_link_text[<a href=#>partial link<br>text</a>-k\nt]
  imported/w3c/webdriver/tests/element_clear/clear.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/find_elements/find.py::test_find_elements_link_text[<a href=#>link<br>text</a>-link\ntext]
  imported/w3c/webdriver/tests/find_elements/find.py::test_find_elements_partial_link_text[<a href=#>partial link<br>text</a>-k\nt]
  imported/w3c/webdriver/tests/bidi/new_session/connect.py::test_websocket_url_connect[capabilities0]
  imported/w3c/webdriver/tests/bidi/new_session/connect.py::test_bidi_session_send
  imported/w3c/webdriver/tests/bidi/new_session/connect.py::test_bidi_session_with_different_capability[capabilities0]
  imported/w3c/webdriver/tests/add_cookie/add.py::test_cookie_unsupported_scheme[about]
  imported/w3c/webdriver/tests/add_cookie/add.py::test_cookie_unsupported_scheme[blob]
  imported/w3c/webdriver/tests/add_cookie/add.py::test_cookie_unsupported_scheme[data]
  imported/w3c/webdriver/tests/add_cookie/add.py::test_cookie_unsupported_scheme[file]
  imported/w3c/webdriver/tests/add_cookie/add.py::test_cookie_unsupported_scheme[ftp]
  imported/w3c/webdriver/tests/add_cookie/add.py::test_cookie_unsupported_scheme[javascript]
  imported/w3c/webdriver/tests/add_cookie/add.py::test_cookie_unsupported_scheme[websocket]
  imported/w3c/webdriver/tests/add_cookie/add.py::test_cookie_unsupported_scheme[secure websocket]
  imported/w3c/webdriver/tests/get_computed_label/get.py
  imported/w3c/webdriver/tests/get_element_tag_name/get.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/maximize_window/maximize.py::test_maximize_twice_is_idempotent
  imported/w3c/webdriver/tests/maximize_window/stress.py::test_stress[0]
  imported/w3c/webdriver/tests/maximize_window/stress.py::test_stress[3]
  imported/w3c/webdriver/tests/maximize_window/stress.py::test_stress[4]
  imported/w3c/webdriver/tests/forward/forward.py::test_cross_origin[capabilities0]
  imported/w3c/webdriver/tests/find_element/find.py::test_find_element_link_text[<a href=#>link<br>text</a>-link\ntext]
  imported/w3c/webdriver/tests/find_element/find.py::test_find_element_partial_link_text[<a href=#>partial link<br>text</a>-k\nt]
  imported/w3c/webdriver/tests/get_element_text/get.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/get_computed_role/get.py
  imported/w3c/webdriver/tests/set_window_rect/set.py::test_x_y_floats
  imported/w3c/webdriver/tests/set_window_rect/set.py::test_x_y
  imported/w3c/webdriver/tests/set_window_rect/set.py::test_x_as_current
  imported/w3c/webdriver/tests/set_window_rect/set.py::test_y_as_current
  imported/w3c/webdriver/tests/set_window_rect/user_prompts.py::test_accept[capabilities0-alert-None]
  imported/w3c/webdriver/tests/set_window_rect/user_prompts.py::test_accept[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/set_window_rect/user_prompts.py::test_accept[capabilities0-prompt-]
  imported/w3c/webdriver/tests/set_window_rect/user_prompts.py::test_dismiss[capabilities0-alert-None]
  imported/w3c/webdriver/tests/set_window_rect/user_prompts.py::test_dismiss[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/set_window_rect/user_prompts.py::test_dismiss[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/release_actions/release.py::test_no_browsing_context
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_null_parameter_value
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_no_browsing_context
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_using_argument[a]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_using_argument[True]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_using_argument[None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_using_argument[1]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_using_argument[using4]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_using_argument[using5]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_selector_argument[None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_selector_argument[value1]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_selector_argument[value2]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_detached_shadow_root
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_equivalence
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements[css selector-#linkText]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements[link text-full link text]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements[partial link text-link text]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements[tag name-a]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements[xpath-//a]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_link_text[<a href=#>link text</a>-link text]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_link_text[<a href=#>&nbsp;link text&nbsp;</a>-link text]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_link_text[<a href=#>link<br>text</a>-link\ntext]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_link_text[<a href=#>link&amp;text</a>-link&text]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_link_text[<a href=#>LINK TEXT</a>-LINK TEXT]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_link_text[<a href=# style='text-transform: uppercase'>link text</a>-LINK TEXT]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_partial_link_text[<a href=#>partial link text</a>-link]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_partial_link_text[<a href=#>&nbsp;partial link text&nbsp;</a>-link]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_partial_link_text[<a href=#>partial link text</a>-k t]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_partial_link_text[<a href=#>partial link<br>text</a>-k\nt]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_partial_link_text[<a href=#>partial link&amp;text</a>-k&t]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_partial_link_text[<a href=#>PARTIAL LINK TEXT</a>-LINK]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_partial_link_text[<a href=# style='text-transform: uppercase'>partial link text</a>-LINK]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_no_element[css selector-#wontExist]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_accept[capabilities0-alert-None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_accept[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_accept[capabilities0-prompt-]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-alert-None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-prompt-]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_dismiss[capabilities0-alert-None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_dismiss[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_dismiss[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-alert-None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_ignore[capabilities0-alert]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_ignore[capabilities0-confirm]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_ignore[capabilities0-prompt]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_default[alert-None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_default[confirm-False]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_default[prompt-None]
  imported/w3c/webdriver/tests/switch_to_frame/switch_webelement.py::test_frame_id_webelement_cloned_into_iframe
  imported/w3c/webdriver/tests/switch_to_frame/switch.py::test_frame_id_null
  imported/w3c/webdriver/tests/get_element_property/get.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/element_click/click.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/element_click/events.py::test_event_mousemove
  imported/w3c/webdriver/tests/get_element_shadow_root/get.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/get_element_shadow_root/get.py::test_no_browsing_context
  imported/w3c/webdriver/tests/get_element_shadow_root/get.py::test_element_not_found
  imported/w3c/webdriver/tests/get_element_shadow_root/get.py::test_element_stale
  imported/w3c/webdriver/tests/get_element_shadow_root/get.py::test_get_shadow_root
  imported/w3c/webdriver/tests/get_element_shadow_root/get.py::test_no_shadow_root
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_accept[capabilities0-alert-None]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_accept[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_accept[capabilities0-prompt-]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-alert-None]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-prompt-]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_dismiss[capabilities0-alert-None]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_dismiss[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_dismiss[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-alert-None]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_ignore[capabilities0-alert]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_ignore[capabilities0-confirm]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_ignore[capabilities0-prompt]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_default[alert-None]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_default[confirm-False]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_default[prompt-None]
  imported/w3c/webdriver/tests/perform_actions/pointer.py::test_pen_pointer_properties
  imported/w3c/webdriver/tests/perform_actions/pointer.py::test_touch_pointer_properties
  imported/w3c/webdriver/tests/perform_actions/key_events.py::test_modifier_key_sends_correct_events[\ue052-R_ALT]
  imported/w3c/webdriver/tests/perform_actions/key_events.py::test_special_key_sends_keydown[R_ALT-expected48]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_null_parameter_value
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_no_browsing_context
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_using_argument[a]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_using_argument[True]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_using_argument[None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_using_argument[1]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_using_argument[using4]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_using_argument[using5]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_selector_argument[None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_selector_argument[value1]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_selector_argument[value2]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_detached_shadow_root
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_found_element_equivalence
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element[css selector-#linkText]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element[link text-full link text]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element[partial link text-link text]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element[tag name-a]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element[xpath-//a]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_link_text[<a href=#>link text</a>-link text]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_link_text[<a href=#>&nbsp;link text&nbsp;</a>-link text]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_link_text[<a href=#>link<br>text</a>-link\ntext]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_link_text[<a href=#>link&amp;text</a>-link&text]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_link_text[<a href=#>LINK TEXT</a>-LINK TEXT]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_link_text[<a href=# style='text-transform: uppercase'>link text</a>-LINK TEXT]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_partial_link_text[<a href=#>partial link text</a>-link]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_partial_link_text[<a href=#>&nbsp;partial link text&nbsp;</a>-link]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_partial_link_text[<a href=#>partial link text</a>-k t]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_partial_link_text[<a href=#>partial link<br>text</a>-k\nt]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_partial_link_text[<a href=#>partial link&amp;text</a>-k&t]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_partial_link_text[<a href=#>PARTIAL LINK TEXT</a>-LINK]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_partial_link_text[<a href=# style='text-transform: uppercase'>partial link text</a>-LINK]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_no_element[css selector-#wontExist]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_accept[capabilities0-alert-None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_accept[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_accept[capabilities0-prompt-]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-alert-None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-prompt-]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_dismiss[capabilities0-alert-None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_dismiss[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_dismiss[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-alert-None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_ignore[capabilities0-alert]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_ignore[capabilities0-confirm]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_ignore[capabilities0-prompt]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_default[alert-None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_default[confirm-False]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_default[prompt-None]
  imported/w3c/webdriver/tests/is_element_selected/selected.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/get_element_rect/get.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/back/back.py::test_cross_origin[capabilities0]
  imported/w3c/webdriver/tests/get_element_css_value/get.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/element_send_keys/send_keys.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/switch_to_window/switch.py::test_element_not_found_after_tab_switch
  imported/w3c/webdriver/tests/get_element_attribute/get.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/get_element_attribute/get.py::test_anchor_href[relative]
  imported/w3c/webdriver/tests/new_session/websocket_url.py::test_websocket_url

Unexpected timeouts (5)
  imported/w3c/webdriver/tests/get_window_handle/get.py::test_navigation_with_coop_headers[capabilities0]
  imported/w3c/webdriver/tests/forward/forward.py::test_cross_origin[capabilities0]
  imported/w3c/webdriver/tests/element_click/navigate.py::test_link_cross_origin[capabilities0]
  imported/w3c/webdriver/tests/navigate_to/navigate.py::test_cross_origin[capabilities0]
  imported/w3c/webdriver/tests/back/back.py::test_cross_origin[capabilities0]


In selenium:

Unexpected failures (20)
  imported/selenium/py/test/alerts_tests.py::testUnexpectedAlertPresentExceptionContainsAlertText[WebKitGTK]
  imported/selenium/py/test/api_example_tests.py::testMoveWindowPosition[WebKitGTK]
  imported/selenium/py/test/interactions_tests.py::testSendingKeysToElement[WebKitGTK]
  imported/selenium/py/test/window_tests.py::test_should_set_the_position_of_the_current_window[WebKitGTK]
  imported/selenium/py/test/window_tests.py::test_should_set_the_rect_of_the_current_window[WebKitGTK]
  imported/selenium/py/test/selenium/alerts_tests.py::testUnexpectedAlertPresentExceptionContainsAlertText[WebKitGTK]
  imported/selenium/py/test/selenium/api_example_tests.py::testMoveWindowPosition[WebKitGTK]
  imported/selenium/py/test/selenium/interactions_tests.py::testSendingKeysToElement[WebKitGTK]
  imported/selenium/py/test/selenium/window_tests.py::test_should_set_the_position_of_the_current_window[WebKitGTK]
  imported/selenium/py/test/selenium/window_tests.py::test_should_set_the_rect_of_the_current_window[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/alerts_tests.py::testUnexpectedAlertPresentExceptionContainsAlertText[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/api_example_tests.py::testMoveWindowPosition[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/interactions_tests.py::testSendingKeysToElement[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/window_tests.py::test_should_set_the_position_of_the_current_window[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/window_tests.py::test_should_set_the_rect_of_the_current_window[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/alerts_tests.py::testUnexpectedAlertPresentExceptionContainsAlertText[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/api_example_tests.py::testMoveWindowPosition[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/window_tests.py::test_should_set_the_position_of_the_current_window[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/window_tests.py::test_should_set_the_rect_of_the_current_window[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/alerts_tests.py::testUnexpectedAlertPresentExceptionContainsAlertText[WebKitGTK]

Unexpected timeouts (4)
  imported/selenium/py/test/frame_switching_tests.py::testShouldFocusOnTheReplacementWhenAFrameFollowsALinkToA_TopTargetedPage[WebKitGTK]
  imported/selenium/py/test/webdriverwait_tests.py::testShouldStillFailToFindAnElementWithExplicitWait[WebKitGTK]
  imported/selenium/py/test/selenium/webdriverwait_tests.py::testShouldStillFailToFindAnElementWithExplicitWait[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/webdriverwait_tests.py::testShouldStillFailToFindAnElementWithExplicitWait[WebKitGTK]

[Carlos Garcia Campos]

The webkitpy failure is probably due to bug #230549

[Carlos Garcia Campos]

hmm, EWS still fails, but now only with python2

[Carlos Garcia Campos]

I've opened bugs for most of the new failures. There are a few remaining that I need to investigate, but we can probably fix or garden them in follow up patches.

[Carlos Garcia Campos]

Created attachment 438943 [details]
Patch

Added test expectations

[Carlos Garcia Campos]

Created attachment 438946 [details]
Patch

[Jonathan Bedard]

Comment on attachment 438946 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=438946&action=review

> Tools/Scripts/webkitpy/__init__.py:74


The Python 2 error is caused by this line, does this module even exist for Python 2? If it doesn't, we need to conditionalize this line on Python 2.

[Lauro Moura]

Got an error about a missing `tools/wptrunner/wptrunner/wptmanifest/backends/base.py` (trace below).

Maybe it's missing from the patch? Re-importing the same hash with the import script shows it as untracked. Manually copying it works around this error (but stops at other import errors I'm still tracking).

Traceback (most recent call last):
  File "Tools/Scripts/run-webdriver-tests", line 30, in <module>
    from webkitpy.webdriver_tests.webdriver_test_runner import WebDriverTestRunner
.....
  File "/home/lauro/dev/WebKit-WPE/WebDriverTests/imported/w3c/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 3, in <module>
    from . import base
ImportError: cannot import name 'base' from 'wptrunner.wptmanifest.backends' (/home/lauro/dev/WebKit-WPE/WebDriverTests/imported/w3c/tools/wptrunner/wptrunner/wptmanifest/backends/__init__.py)

[Lauro Moura]

The remaining ImportError:

  File "/home/lauro/dev/WebKit-WPE/WebDriverTests/imported/w3c/tools/wptrunner/wptrunner/webdriver_server.py", line 12, in <module>
    from .browsers.base import OutputHandler
  File "/home/lauro/dev/WebKit-WPE/WebDriverTests/imported/w3c/tools/wptrunner/wptrunner/browsers/base.py", line 7, in <module>
    from ..wptcommandline import require_arg  # noqa: F401
  File "/home/lauro/dev/WebKit-WPE/WebDriverTests/imported/w3c/tools/wptrunner/wptrunner/wptcommandline.py", line 10, in <module>
    from .formatters import chromium, wptreport, wptscreenshot
ImportError: cannot import name 'chromium' from 'wptrunner.formatters' (/home/lauro/dev/WebKit-WPE/WebDriverTests/imported/w3c/tools/wptrunner/wptrunner/formatters.py)

web_driver_server.py changes introduced this new browser.base import.

Removing the formatters imports (and respective usage inside wptcommandline.py, the test suite runs normally.

[Lauro Moura]

Sorry, there was a small issue left at the end of the test run. webdriver_test_runner.py trying to write json to a 'wb' opened file instead of 'w' for py3:

Traceback (most recent call last):
  File "/app/webkit/Tools/Scripts/run-webdriver-tests", line 90, in <module>
    runner.dump_results_to_json_file(options.json_output)
  File "/app/webkit/Tools/Scripts/webkitpy/webdriver_tests/webdriver_test_runner.py", line 169, in dump_results_to_json_file
    json.dump(json_results, fp)
  File "/usr/lib/python3.8/json/__init__.py", line 180, in dump
    fp.write(chunk)
TypeError: a bytes-like object is required, not 'str'

[Carlos Garcia Campos]

Comment on attachment 438946 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=438946&action=review

>> Tools/Scripts/webkitpy/__init__.py:74
>> +AutoInstall.register(Package('websockets', Version(8, 1)))
> 
> The Python 2 error is caused by this line, does this module even exist for Python 2? If it doesn't, we need to conditionalize this line on Python 2.

That's the issue, it requires python3, but it's only needed by pytest that will always be used with python3 after this patch.

[Carlos Garcia Campos]

(In reply to Lauro Moura from comment #9)
> Got an error about a missing
> `tools/wptrunner/wptrunner/wptmanifest/backends/base.py` (trace below).

weird, tests run here.

> Maybe it's missing from the patch? Re-importing the same hash with the
> import script shows it as untracked. Manually copying it works around this
> error (but stops at other import errors I'm still tracking).

hmm, I'll check

> Traceback (most recent call last):
>   File "Tools/Scripts/run-webdriver-tests", line 30, in <module>
>     from webkitpy.webdriver_tests.webdriver_test_runner import
> WebDriverTestRunner
> .....
>   File
> "/home/lauro/dev/WebKit-WPE/WebDriverTests/imported/w3c/tools/wptrunner/
> wptrunner/wptmanifest/backends/static.py", line 3, in <module>
>     from . import base
> ImportError: cannot import name 'base' from 'wptrunner.wptmanifest.backends'
> (/home/lauro/dev/WebKit-WPE/WebDriverTests/imported/w3c/tools/wptrunner/
> wptrunner/wptmanifest/backends/__init__.py)

[Carlos Garcia Campos]

Created attachment 439024 [details]
Patch

Added missing files and made websockets python3 only.

[Carlos Garcia Campos]

Created attachment 439035 [details]
Patch

[Lauro Moura]

(In reply to Carlos Garcia Campos from comment #15)
> Created attachment 439035 [details]
> Patch

Working fine for GTK here.

For WPE, running the following test with Cog makes the test suite to hang. With MiniBrowser it timeouts without getting stuck, like GTK. As the tests currently are still run with MiniBrowser, this should not be a blocker.


 * imported/w3c/webdriver/tests/forward/forward.py::test_cross_origin[capabilities0]

[Jonathan Bedard]

Comment on attachment 439035 [details]
Patch

Do not have any objections to this change, but I also won't be able to review it until next week.

[Lauro Moura]

(In reply to Lauro Moura from comment #16)
> 
> Working fine for GTK here.
> 
> For WPE, running the following test with Cog makes the test suite to hang.
> With MiniBrowser it timeouts without getting stuck, like GTK. As the tests
> currently are still run with MiniBrowser, this should not be a blocker.
> 
> 
>  *
> imported/w3c/webdriver/tests/forward/forward.py::
> test_cross_origin[capabilities0]

The issue with Cog is that during this test the WebDriver loses connection with the browser and retries to launch it during the test teardown. As the previous instance is still running, the newly created Cog's GApplication just returns normally. Somehow this early return is making the WebDriver/test suite to hang instead of timing out.

Tomorrow I'll open a detailed bug for it.

[Radar WebKit Bug Importer]

<rdar://problem/83620518>

[Carlos Garcia Campos]

Ping reviewers.

[Jonathan Bedard]

(In reply to Carlos Garcia Campos from comment #20)
> Ping reviewers.

I this change Python 2 compatible, or do we need to switch over all the bots when it lands?

[Carlos Garcia Campos]

(In reply to Jonathan Bedard from comment #21)
> (In reply to Carlos Garcia Campos from comment #20)
> > Ping reviewers.
> 
> I this change Python 2 compatible, or do we need to switch over all the bots
> when it lands?

hmm, webdriver test runner will now use python3 always, I think. I guess that's not a problem for the bots.

[Jonathan Bedard]

(In reply to Carlos Garcia Campos from comment #22)
> (In reply to Jonathan Bedard from comment #21)
> > (In reply to Carlos Garcia Campos from comment #20)
> > > Ping reviewers.
> > 
> > I this change Python 2 compatible, or do we need to switch over all the bots
> > when it lands?
> 
> hmm, webdriver test runner will now use python3 always, I think. I guess
> that's not a problem for the bots.

Buildbot specifically invokes this script with "python", see Tools/CISupport/build-webkit-org/steps.py. It doesn't respect the she-bang.

[Carlos Garcia Campos]

(In reply to Jonathan Bedard from comment #23)
> (In reply to Carlos Garcia Campos from comment #22)
> > (In reply to Jonathan Bedard from comment #21)
> > > (In reply to Carlos Garcia Campos from comment #20)
> > > > Ping reviewers.
> > > 
> > > I this change Python 2 compatible, or do we need to switch over all the bots
> > > when it lands?
> > 
> > hmm, webdriver test runner will now use python3 always, I think. I guess
> > that's not a problem for the bots.
> 
> Buildbot specifically invokes this script with "python", see
> Tools/CISupport/build-webkit-org/steps.py. It doesn't respect the she-bang.

Ah! we need to change that then.

[Carlos Garcia Campos]

Created attachment 440486 [details]
fast-cq-Patch

[Jonathan Bedard]

Comment on attachment 440486 [details]
fast-cq-Patch

r+ing, but holding off on commit-queue until Carlos and I coordinate landing with restarting build.webkit.org

[EWS]

Committed r283798 (242691@main): <https://commits.webkit.org/242691@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 440486 [details].

[Ryan Haddad]

*** Bug 230097 has been marked as a duplicate of this bug. ***