Bug 230551 - [WebDriver] Update w3c and selenium tests
Summary: [WebDriver] Update w3c and selenium tests
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebDriver (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on: 230549
Blocks:
  Show dependency treegraph
 
Reported: 2021-09-21 06:48 PDT by Carlos Garcia Campos
Modified: 2021-10-08 07:21 PDT (History)
9 users (show)

See Also:


Attachments
Patch (707.13 KB, patch)
2021-09-21 06:58 PDT, Carlos Garcia Campos
no flags Details | Formatted Diff | Diff
Patch (716.67 KB, patch)
2021-09-22 06:14 PDT, Carlos Garcia Campos
no flags Details | Formatted Diff | Diff
Patch (720.74 KB, patch)
2021-09-22 07:09 PDT, Carlos Garcia Campos
no flags Details | Formatted Diff | Diff
Patch (787.52 KB, patch)
2021-09-23 01:48 PDT, Carlos Garcia Campos
no flags Details | Formatted Diff | Diff
Patch (787.62 KB, patch)
2021-09-23 04:20 PDT, Carlos Garcia Campos
no flags Details | Formatted Diff | Diff
fast-cq-Patch (788.58 KB, patch)
2021-10-07 01:08 PDT, Carlos Garcia Campos
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Carlos Garcia Campos 2021-09-21 06:48:56 PDT
It's been a while since the last time we imported w3c and selenium tests. They both now require python3 and a new version of pytest, so it requires changes in our test runner too.
Comment 1 Carlos Garcia Campos 2021-09-21 06:58:28 PDT
Created attachment 438811 [details]
Patch
Comment 2 Carlos Garcia Campos 2021-09-21 07:02:49 PDT
I still have to do the gardening, there are a lot of new failures...

In w3c:

Unexpected failures (186)
  imported/w3c/webdriver/tests/switch_to_parent_frame/switch.py::test_switch_from_iframe
  imported/w3c/webdriver/tests/find_elements_from_element/find.py::test_find_elements_link_text[<a href=#>link<br>text</a>-link\ntext]
  imported/w3c/webdriver/tests/find_elements_from_element/find.py::test_find_elements_partial_link_text[<a href=#>partial link<br>text</a>-k\nt]
  imported/w3c/webdriver/tests/is_element_enabled/enabled.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/find_element_from_element/find.py::test_find_element_link_text[<a href=#>link<br>text</a>-link\ntext]
  imported/w3c/webdriver/tests/find_element_from_element/find.py::test_find_element_partial_link_text[<a href=#>partial link<br>text</a>-k\nt]
  imported/w3c/webdriver/tests/element_clear/clear.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/find_elements/find.py::test_find_elements_link_text[<a href=#>link<br>text</a>-link\ntext]
  imported/w3c/webdriver/tests/find_elements/find.py::test_find_elements_partial_link_text[<a href=#>partial link<br>text</a>-k\nt]
  imported/w3c/webdriver/tests/bidi/new_session/connect.py::test_websocket_url_connect[capabilities0]
  imported/w3c/webdriver/tests/bidi/new_session/connect.py::test_bidi_session_send
  imported/w3c/webdriver/tests/bidi/new_session/connect.py::test_bidi_session_with_different_capability[capabilities0]
  imported/w3c/webdriver/tests/add_cookie/add.py::test_cookie_unsupported_scheme[about]
  imported/w3c/webdriver/tests/add_cookie/add.py::test_cookie_unsupported_scheme[blob]
  imported/w3c/webdriver/tests/add_cookie/add.py::test_cookie_unsupported_scheme[data]
  imported/w3c/webdriver/tests/add_cookie/add.py::test_cookie_unsupported_scheme[file]
  imported/w3c/webdriver/tests/add_cookie/add.py::test_cookie_unsupported_scheme[ftp]
  imported/w3c/webdriver/tests/add_cookie/add.py::test_cookie_unsupported_scheme[javascript]
  imported/w3c/webdriver/tests/add_cookie/add.py::test_cookie_unsupported_scheme[websocket]
  imported/w3c/webdriver/tests/add_cookie/add.py::test_cookie_unsupported_scheme[secure websocket]
  imported/w3c/webdriver/tests/get_computed_label/get.py
  imported/w3c/webdriver/tests/get_element_tag_name/get.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/maximize_window/maximize.py::test_maximize_twice_is_idempotent
  imported/w3c/webdriver/tests/maximize_window/stress.py::test_stress[0]
  imported/w3c/webdriver/tests/maximize_window/stress.py::test_stress[3]
  imported/w3c/webdriver/tests/maximize_window/stress.py::test_stress[4]
  imported/w3c/webdriver/tests/forward/forward.py::test_cross_origin[capabilities0]
  imported/w3c/webdriver/tests/find_element/find.py::test_find_element_link_text[<a href=#>link<br>text</a>-link\ntext]
  imported/w3c/webdriver/tests/find_element/find.py::test_find_element_partial_link_text[<a href=#>partial link<br>text</a>-k\nt]
  imported/w3c/webdriver/tests/get_element_text/get.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/get_computed_role/get.py
  imported/w3c/webdriver/tests/set_window_rect/set.py::test_x_y_floats
  imported/w3c/webdriver/tests/set_window_rect/set.py::test_x_y
  imported/w3c/webdriver/tests/set_window_rect/set.py::test_x_as_current
  imported/w3c/webdriver/tests/set_window_rect/set.py::test_y_as_current
  imported/w3c/webdriver/tests/set_window_rect/user_prompts.py::test_accept[capabilities0-alert-None]
  imported/w3c/webdriver/tests/set_window_rect/user_prompts.py::test_accept[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/set_window_rect/user_prompts.py::test_accept[capabilities0-prompt-]
  imported/w3c/webdriver/tests/set_window_rect/user_prompts.py::test_dismiss[capabilities0-alert-None]
  imported/w3c/webdriver/tests/set_window_rect/user_prompts.py::test_dismiss[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/set_window_rect/user_prompts.py::test_dismiss[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/release_actions/release.py::test_no_browsing_context
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_null_parameter_value
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_no_browsing_context
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_using_argument[a]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_using_argument[True]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_using_argument[None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_using_argument[1]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_using_argument[using4]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_using_argument[using5]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_selector_argument[None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_selector_argument[value1]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_invalid_selector_argument[value2]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_detached_shadow_root
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_equivalence
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements[css selector-#linkText]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements[link text-full link text]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements[partial link text-link text]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements[tag name-a]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements[xpath-//a]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_link_text[<a href=#>link text</a>-link text]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_link_text[<a href=#>&nbsp;link text&nbsp;</a>-link text]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_link_text[<a href=#>link<br>text</a>-link\ntext]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_link_text[<a href=#>link&amp;text</a>-link&text]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_link_text[<a href=#>LINK TEXT</a>-LINK TEXT]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_link_text[<a href=# style='text-transform: uppercase'>link text</a>-LINK TEXT]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_partial_link_text[<a href=#>partial link text</a>-link]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_partial_link_text[<a href=#>&nbsp;partial link text&nbsp;</a>-link]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_partial_link_text[<a href=#>partial link text</a>-k t]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_partial_link_text[<a href=#>partial link<br>text</a>-k\nt]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_partial_link_text[<a href=#>partial link&amp;text</a>-k&t]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_partial_link_text[<a href=#>PARTIAL LINK TEXT</a>-LINK]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_find_elements_partial_link_text[<a href=# style='text-transform: uppercase'>partial link text</a>-LINK]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/find.py::test_no_element[css selector-#wontExist]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_accept[capabilities0-alert-None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_accept[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_accept[capabilities0-prompt-]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-alert-None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-prompt-]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_dismiss[capabilities0-alert-None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_dismiss[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_dismiss[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-alert-None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_ignore[capabilities0-alert]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_ignore[capabilities0-confirm]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_ignore[capabilities0-prompt]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_default[alert-None]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_default[confirm-False]
  imported/w3c/webdriver/tests/find_elements_from_shadow_root/user_prompts.py::test_default[prompt-None]
  imported/w3c/webdriver/tests/switch_to_frame/switch_webelement.py::test_frame_id_webelement_cloned_into_iframe
  imported/w3c/webdriver/tests/switch_to_frame/switch.py::test_frame_id_null
  imported/w3c/webdriver/tests/get_element_property/get.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/element_click/click.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/element_click/events.py::test_event_mousemove
  imported/w3c/webdriver/tests/get_element_shadow_root/get.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/get_element_shadow_root/get.py::test_no_browsing_context
  imported/w3c/webdriver/tests/get_element_shadow_root/get.py::test_element_not_found
  imported/w3c/webdriver/tests/get_element_shadow_root/get.py::test_element_stale
  imported/w3c/webdriver/tests/get_element_shadow_root/get.py::test_get_shadow_root
  imported/w3c/webdriver/tests/get_element_shadow_root/get.py::test_no_shadow_root
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_accept[capabilities0-alert-None]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_accept[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_accept[capabilities0-prompt-]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-alert-None]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-prompt-]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_dismiss[capabilities0-alert-None]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_dismiss[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_dismiss[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-alert-None]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_ignore[capabilities0-alert]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_ignore[capabilities0-confirm]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_ignore[capabilities0-prompt]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_default[alert-None]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_default[confirm-False]
  imported/w3c/webdriver/tests/get_element_shadow_root/user_prompts.py::test_default[prompt-None]
  imported/w3c/webdriver/tests/perform_actions/pointer.py::test_pen_pointer_properties
  imported/w3c/webdriver/tests/perform_actions/pointer.py::test_touch_pointer_properties
  imported/w3c/webdriver/tests/perform_actions/key_events.py::test_modifier_key_sends_correct_events[\ue052-R_ALT]
  imported/w3c/webdriver/tests/perform_actions/key_events.py::test_special_key_sends_keydown[R_ALT-expected48]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_null_parameter_value
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_no_browsing_context
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_using_argument[a]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_using_argument[True]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_using_argument[None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_using_argument[1]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_using_argument[using4]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_using_argument[using5]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_selector_argument[None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_selector_argument[value1]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_invalid_selector_argument[value2]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_detached_shadow_root
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_found_element_equivalence
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element[css selector-#linkText]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element[link text-full link text]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element[partial link text-link text]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element[tag name-a]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element[xpath-//a]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_link_text[<a href=#>link text</a>-link text]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_link_text[<a href=#>&nbsp;link text&nbsp;</a>-link text]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_link_text[<a href=#>link<br>text</a>-link\ntext]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_link_text[<a href=#>link&amp;text</a>-link&text]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_link_text[<a href=#>LINK TEXT</a>-LINK TEXT]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_link_text[<a href=# style='text-transform: uppercase'>link text</a>-LINK TEXT]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_partial_link_text[<a href=#>partial link text</a>-link]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_partial_link_text[<a href=#>&nbsp;partial link text&nbsp;</a>-link]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_partial_link_text[<a href=#>partial link text</a>-k t]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_partial_link_text[<a href=#>partial link<br>text</a>-k\nt]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_partial_link_text[<a href=#>partial link&amp;text</a>-k&t]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_partial_link_text[<a href=#>PARTIAL LINK TEXT</a>-LINK]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_find_element_partial_link_text[<a href=# style='text-transform: uppercase'>partial link text</a>-LINK]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/find.py::test_no_element[css selector-#wontExist]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_accept[capabilities0-alert-None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_accept[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_accept[capabilities0-prompt-]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-alert-None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_accept_and_notify[capabilities0-prompt-]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_dismiss[capabilities0-alert-None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_dismiss[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_dismiss[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-alert-None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_dismiss_and_notify[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_ignore[capabilities0-alert]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_ignore[capabilities0-confirm]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_ignore[capabilities0-prompt]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_default[alert-None]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_default[confirm-False]
  imported/w3c/webdriver/tests/find_element_from_shadow_root/user_prompts.py::test_default[prompt-None]
  imported/w3c/webdriver/tests/is_element_selected/selected.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/get_element_rect/get.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/back/back.py::test_cross_origin[capabilities0]
  imported/w3c/webdriver/tests/get_element_css_value/get.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/element_send_keys/send_keys.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/switch_to_window/switch.py::test_element_not_found_after_tab_switch
  imported/w3c/webdriver/tests/get_element_attribute/get.py::test_no_top_browsing_context
  imported/w3c/webdriver/tests/get_element_attribute/get.py::test_anchor_href[relative]
  imported/w3c/webdriver/tests/new_session/websocket_url.py::test_websocket_url

Unexpected timeouts (5)
  imported/w3c/webdriver/tests/get_window_handle/get.py::test_navigation_with_coop_headers[capabilities0]
  imported/w3c/webdriver/tests/forward/forward.py::test_cross_origin[capabilities0]
  imported/w3c/webdriver/tests/element_click/navigate.py::test_link_cross_origin[capabilities0]
  imported/w3c/webdriver/tests/navigate_to/navigate.py::test_cross_origin[capabilities0]
  imported/w3c/webdriver/tests/back/back.py::test_cross_origin[capabilities0]


In selenium:

Unexpected failures (20)
  imported/selenium/py/test/alerts_tests.py::testUnexpectedAlertPresentExceptionContainsAlertText[WebKitGTK]
  imported/selenium/py/test/api_example_tests.py::testMoveWindowPosition[WebKitGTK]
  imported/selenium/py/test/interactions_tests.py::testSendingKeysToElement[WebKitGTK]
  imported/selenium/py/test/window_tests.py::test_should_set_the_position_of_the_current_window[WebKitGTK]
  imported/selenium/py/test/window_tests.py::test_should_set_the_rect_of_the_current_window[WebKitGTK]
  imported/selenium/py/test/selenium/alerts_tests.py::testUnexpectedAlertPresentExceptionContainsAlertText[WebKitGTK]
  imported/selenium/py/test/selenium/api_example_tests.py::testMoveWindowPosition[WebKitGTK]
  imported/selenium/py/test/selenium/interactions_tests.py::testSendingKeysToElement[WebKitGTK]
  imported/selenium/py/test/selenium/window_tests.py::test_should_set_the_position_of_the_current_window[WebKitGTK]
  imported/selenium/py/test/selenium/window_tests.py::test_should_set_the_rect_of_the_current_window[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/alerts_tests.py::testUnexpectedAlertPresentExceptionContainsAlertText[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/api_example_tests.py::testMoveWindowPosition[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/interactions_tests.py::testSendingKeysToElement[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/window_tests.py::test_should_set_the_position_of_the_current_window[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/window_tests.py::test_should_set_the_rect_of_the_current_window[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/alerts_tests.py::testUnexpectedAlertPresentExceptionContainsAlertText[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/api_example_tests.py::testMoveWindowPosition[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/window_tests.py::test_should_set_the_position_of_the_current_window[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/window_tests.py::test_should_set_the_rect_of_the_current_window[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/alerts_tests.py::testUnexpectedAlertPresentExceptionContainsAlertText[WebKitGTK]

Unexpected timeouts (4)
  imported/selenium/py/test/frame_switching_tests.py::testShouldFocusOnTheReplacementWhenAFrameFollowsALinkToA_TopTargetedPage[WebKitGTK]
  imported/selenium/py/test/webdriverwait_tests.py::testShouldStillFailToFindAnElementWithExplicitWait[WebKitGTK]
  imported/selenium/py/test/selenium/webdriverwait_tests.py::testShouldStillFailToFindAnElementWithExplicitWait[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/webdriverwait_tests.py::testShouldStillFailToFindAnElementWithExplicitWait[WebKitGTK]
Comment 3 Carlos Garcia Campos 2021-09-21 07:04:34 PDT
The webkitpy failure is probably due to bug #230549
Comment 4 Carlos Garcia Campos 2021-09-22 04:17:48 PDT
hmm, EWS still fails, but now only with python2
Comment 5 Carlos Garcia Campos 2021-09-22 05:43:10 PDT
I've opened bugs for most of the new failures. There are a few remaining that I need to investigate, but we can probably fix or garden them in follow up patches.
Comment 6 Carlos Garcia Campos 2021-09-22 06:14:14 PDT
Created attachment 438943 [details]
Patch

Added test expectations
Comment 7 Carlos Garcia Campos 2021-09-22 07:09:32 PDT
Created attachment 438946 [details]
Patch
Comment 8 Jonathan Bedard 2021-09-22 07:49:06 PDT
Comment on attachment 438946 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=438946&action=review

> Tools/Scripts/webkitpy/__init__.py:74


The Python 2 error is caused by this line, does this module even exist for Python 2? If it doesn't, we need to conditionalize this line on Python 2.
Comment 9 Lauro Moura 2021-09-22 15:12:41 PDT
Got an error about a missing `tools/wptrunner/wptrunner/wptmanifest/backends/base.py` (trace below).

Maybe it's missing from the patch? Re-importing the same hash with the import script shows it as untracked. Manually copying it works around this error (but stops at other import errors I'm still tracking).

Traceback (most recent call last):
  File "Tools/Scripts/run-webdriver-tests", line 30, in <module>
    from webkitpy.webdriver_tests.webdriver_test_runner import WebDriverTestRunner
.....
  File "/home/lauro/dev/WebKit-WPE/WebDriverTests/imported/w3c/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 3, in <module>
    from . import base
ImportError: cannot import name 'base' from 'wptrunner.wptmanifest.backends' (/home/lauro/dev/WebKit-WPE/WebDriverTests/imported/w3c/tools/wptrunner/wptrunner/wptmanifest/backends/__init__.py)
Comment 10 Lauro Moura 2021-09-22 19:27:43 PDT
The remaining ImportError:

  File "/home/lauro/dev/WebKit-WPE/WebDriverTests/imported/w3c/tools/wptrunner/wptrunner/webdriver_server.py", line 12, in <module>
    from .browsers.base import OutputHandler
  File "/home/lauro/dev/WebKit-WPE/WebDriverTests/imported/w3c/tools/wptrunner/wptrunner/browsers/base.py", line 7, in <module>
    from ..wptcommandline import require_arg  # noqa: F401
  File "/home/lauro/dev/WebKit-WPE/WebDriverTests/imported/w3c/tools/wptrunner/wptrunner/wptcommandline.py", line 10, in <module>
    from .formatters import chromium, wptreport, wptscreenshot
ImportError: cannot import name 'chromium' from 'wptrunner.formatters' (/home/lauro/dev/WebKit-WPE/WebDriverTests/imported/w3c/tools/wptrunner/wptrunner/formatters.py)

web_driver_server.py changes introduced this new browser.base import.

Removing the formatters imports (and respective usage inside wptcommandline.py, the test suite runs normally.
Comment 11 Lauro Moura 2021-09-22 19:42:07 PDT
Sorry, there was a small issue left at the end of the test run. webdriver_test_runner.py trying to write json to a 'wb' opened file instead of 'w' for py3:

Traceback (most recent call last):
  File "/app/webkit/Tools/Scripts/run-webdriver-tests", line 90, in <module>
    runner.dump_results_to_json_file(options.json_output)
  File "/app/webkit/Tools/Scripts/webkitpy/webdriver_tests/webdriver_test_runner.py", line 169, in dump_results_to_json_file
    json.dump(json_results, fp)
  File "/usr/lib/python3.8/json/__init__.py", line 180, in dump
    fp.write(chunk)
TypeError: a bytes-like object is required, not 'str'
Comment 12 Carlos Garcia Campos 2021-09-23 01:35:25 PDT
Comment on attachment 438946 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=438946&action=review

>> Tools/Scripts/webkitpy/__init__.py:74
>> +AutoInstall.register(Package('websockets', Version(8, 1)))
> 
> The Python 2 error is caused by this line, does this module even exist for Python 2? If it doesn't, we need to conditionalize this line on Python 2.

That's the issue, it requires python3, but it's only needed by pytest that will always be used with python3 after this patch.
Comment 13 Carlos Garcia Campos 2021-09-23 01:36:36 PDT
(In reply to Lauro Moura from comment #9)
> Got an error about a missing
> `tools/wptrunner/wptrunner/wptmanifest/backends/base.py` (trace below).

weird, tests run here.

> Maybe it's missing from the patch? Re-importing the same hash with the
> import script shows it as untracked. Manually copying it works around this
> error (but stops at other import errors I'm still tracking).

hmm, I'll check

> Traceback (most recent call last):
>   File "Tools/Scripts/run-webdriver-tests", line 30, in <module>
>     from webkitpy.webdriver_tests.webdriver_test_runner import
> WebDriverTestRunner
> .....
>   File
> "/home/lauro/dev/WebKit-WPE/WebDriverTests/imported/w3c/tools/wptrunner/
> wptrunner/wptmanifest/backends/static.py", line 3, in <module>
>     from . import base
> ImportError: cannot import name 'base' from 'wptrunner.wptmanifest.backends'
> (/home/lauro/dev/WebKit-WPE/WebDriverTests/imported/w3c/tools/wptrunner/
> wptrunner/wptmanifest/backends/__init__.py)
Comment 14 Carlos Garcia Campos 2021-09-23 01:48:43 PDT
Created attachment 439024 [details]
Patch

Added missing files and made websockets python3 only.
Comment 15 Carlos Garcia Campos 2021-09-23 04:20:33 PDT
Created attachment 439035 [details]
Patch
Comment 16 Lauro Moura 2021-09-23 07:44:08 PDT
(In reply to Carlos Garcia Campos from comment #15)
> Created attachment 439035 [details]
> Patch

Working fine for GTK here.

For WPE, running the following test with Cog makes the test suite to hang. With MiniBrowser it timeouts without getting stuck, like GTK. As the tests currently are still run with MiniBrowser, this should not be a blocker.


 * imported/w3c/webdriver/tests/forward/forward.py::test_cross_origin[capabilities0]
Comment 17 Jonathan Bedard 2021-09-23 08:15:18 PDT
Comment on attachment 439035 [details]
Patch

Do not have any objections to this change, but I also won't be able to review it until next week.
Comment 18 Lauro Moura 2021-09-23 21:55:43 PDT
(In reply to Lauro Moura from comment #16)
> 
> Working fine for GTK here.
> 
> For WPE, running the following test with Cog makes the test suite to hang.
> With MiniBrowser it timeouts without getting stuck, like GTK. As the tests
> currently are still run with MiniBrowser, this should not be a blocker.
> 
> 
>  *
> imported/w3c/webdriver/tests/forward/forward.py::
> test_cross_origin[capabilities0]

The issue with Cog is that during this test the WebDriver loses connection with the browser and retries to launch it during the test teardown. As the previous instance is still running, the newly created Cog's GApplication just returns normally. Somehow this early return is making the WebDriver/test suite to hang instead of timing out.

Tomorrow I'll open a detailed bug for it.
Comment 19 Radar WebKit Bug Importer 2021-09-28 06:49:16 PDT
<rdar://problem/83620518>
Comment 20 Carlos Garcia Campos 2021-09-28 06:54:18 PDT
Ping reviewers.
Comment 21 Jonathan Bedard 2021-10-06 08:14:52 PDT
(In reply to Carlos Garcia Campos from comment #20)
> Ping reviewers.

I this change Python 2 compatible, or do we need to switch over all the bots when it lands?
Comment 22 Carlos Garcia Campos 2021-10-06 08:17:48 PDT
(In reply to Jonathan Bedard from comment #21)
> (In reply to Carlos Garcia Campos from comment #20)
> > Ping reviewers.
> 
> I this change Python 2 compatible, or do we need to switch over all the bots
> when it lands?

hmm, webdriver test runner will now use python3 always, I think. I guess that's not a problem for the bots.
Comment 23 Jonathan Bedard 2021-10-06 08:44:53 PDT
(In reply to Carlos Garcia Campos from comment #22)
> (In reply to Jonathan Bedard from comment #21)
> > (In reply to Carlos Garcia Campos from comment #20)
> > > Ping reviewers.
> > 
> > I this change Python 2 compatible, or do we need to switch over all the bots
> > when it lands?
> 
> hmm, webdriver test runner will now use python3 always, I think. I guess
> that's not a problem for the bots.

Buildbot specifically invokes this script with "python", see Tools/CISupport/build-webkit-org/steps.py. It doesn't respect the she-bang.
Comment 24 Carlos Garcia Campos 2021-10-07 00:44:10 PDT
(In reply to Jonathan Bedard from comment #23)
> (In reply to Carlos Garcia Campos from comment #22)
> > (In reply to Jonathan Bedard from comment #21)
> > > (In reply to Carlos Garcia Campos from comment #20)
> > > > Ping reviewers.
> > > 
> > > I this change Python 2 compatible, or do we need to switch over all the bots
> > > when it lands?
> > 
> > hmm, webdriver test runner will now use python3 always, I think. I guess
> > that's not a problem for the bots.
> 
> Buildbot specifically invokes this script with "python", see
> Tools/CISupport/build-webkit-org/steps.py. It doesn't respect the she-bang.

Ah! we need to change that then.
Comment 25 Carlos Garcia Campos 2021-10-07 01:08:20 PDT
Created attachment 440486 [details]
fast-cq-Patch
Comment 26 Jonathan Bedard 2021-10-07 07:28:40 PDT
Comment on attachment 440486 [details]
fast-cq-Patch

r+ing, but holding off on commit-queue until Carlos and I coordinate landing with restarting build.webkit.org
Comment 27 EWS 2021-10-08 07:21:43 PDT
Committed r283798 (242691@main): <https://commits.webkit.org/242691@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 440486 [details].