Bug 210229 – Unreviewed gardening. Update expectations of several WebDriver tests

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Diego Pino]

[GTK] Gardening, WebDriver tests expected to fail but passed

[Diego Pino]

Created attachment 395891 [details]
Patch

[Diego Pino]

The GTK test bot reports the following WebDriver tests as expected to fail but passed (https://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release%20%28Tests%29/builds/13254/steps/webdriver-test/logs/stdio):

Expected to fail, but passed (86)
  imported/selenium/py/test/selenium/webdriver/common/visibility_tests.py::testShouldNotBeAbleToTypeAnElementThatIsNotDisplayed[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/interactions_tests.py::testSendingKeysToActiveElementWithModifier[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/click_scrolling_tests.py::testShouldBeAbleToClickOnAnElementHiddenByDoubleOverflow[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/click_scrolling_tests.py::testShouldBeAbleToClickElementThatIsOutOfViewInAFrame[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/click_scrolling_tests.py::testShouldBeAbleToClickElementThatIsOutOfViewInANestedFrame[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/w3c_interaction_tests.py::testSendingKeysToActiveElementWithModifier[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/window_switching_tests.py::testClickingOnAButtonThatClosesAnOpenWindowDoesNotCauseTheBrowserToHang[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/text_handling_tests.py::testHavingInlineElementsShouldNotAffectHowTextIsReturned[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/text_handling_tests.py::testReadALargeAmountOfData[WebKitGTK]
  imported/selenium/py/test/selenium/webdriver/common/correct_event_firing_tests.py::testClearingAnElementShouldCauseTheOnChangeHandlerToFire[WebKitGTK]
  imported/w3c/webdriver/tests/fullscreen_window/user_prompts.py::test_accept[capabilities0-alert-None]
  imported/w3c/webdriver/tests/fullscreen_window/user_prompts.py::test_accept[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/fullscreen_window/user_prompts.py::test_accept[capabilities0-prompt-]
  imported/w3c/webdriver/tests/fullscreen_window/user_prompts.py::test_dismiss[capabilities0-alert-None]
  imported/w3c/webdriver/tests/fullscreen_window/user_prompts.py::test_dismiss[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/fullscreen_window/user_prompts.py::test_dismiss[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/fullscreen_window/stress.py::test_stress[0]
  imported/w3c/webdriver/tests/fullscreen_window/stress.py::test_stress[1]
  imported/w3c/webdriver/tests/fullscreen_window/stress.py::test_stress[2]
  imported/w3c/webdriver/tests/fullscreen_window/stress.py::test_stress[3]
  imported/w3c/webdriver/tests/fullscreen_window/stress.py::test_stress[4]
  imported/w3c/webdriver/tests/fullscreen_window/fullscreen.py::test_fullscreen
  imported/w3c/webdriver/tests/fullscreen_window/fullscreen.py::test_payload
  imported/w3c/webdriver/tests/fullscreen_window/fullscreen.py::test_fullscreen_twice_is_idempotent
  imported/w3c/webdriver/tests/set_window_rect/set.py::test_fully_exit_fullscreen
  imported/w3c/webdriver/tests/get_element_property/get.py::test_element_not_found
  imported/w3c/webdriver/tests/get_element_text/get.py::test_getting_text_of_a_non_existant_element_is_an_error
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_accept[capabilities0-alert-None]
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_accept[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_accept[capabilities0-prompt-]
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_accept_and_notify[capabilities0-alert-None]
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_accept_and_notify[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_accept_and_notify[capabilities0-prompt-]
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_dismiss[capabilities0-alert-None]
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_dismiss[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_dismiss[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_dismiss_and_notify[capabilities0-alert-None]
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_dismiss_and_notify[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_dismiss_and_notify[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_ignore[capabilities0-alert]
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_ignore[capabilities0-confirm]
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_ignore[capabilities0-prompt]
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_default[alert-None]
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_default[confirm-False]
  imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_default[prompt-None]
  imported/w3c/webdriver/tests/element_send_keys/form_controls.py::test_textarea_append
  imported/w3c/webdriver/tests/maximize_window/maximize.py::test_fully_exit_fullscreen
  imported/w3c/webdriver/tests/element_click/bubbling.py::test_spin_event_loop
  imported/w3c/webdriver/tests/element_click/interactability.py::test_display_none
  imported/w3c/webdriver/tests/element_click/interactability.py::test_visibility_hidden
  imported/w3c/webdriver/tests/element_click/interactability.py::test_hidden
  imported/w3c/webdriver/tests/element_click/interactability.py::test_disabled
  imported/w3c/webdriver/tests/element_click/interactability.py::test_element_not_interactable_css_transform[translate(-100px, -100px)]
  imported/w3c/webdriver/tests/element_click/interactability.py::test_element_not_interactable_css_transform[rotate(50deg)]
  imported/w3c/webdriver/tests/element_click/interactability.py::test_element_not_interactable_out_of_view
  imported/w3c/webdriver/tests/element_click/interactability.py::test_zero_sized_element[div]
  imported/w3c/webdriver/tests/element_click/interactability.py::test_zero_sized_element[span]
  imported/w3c/webdriver/tests/perform_actions/pointer_origin.py::test_element_larger_than_viewport
  imported/w3c/webdriver/tests/perform_actions/key_events.py::test_special_key_sends_keydown[PAGE_UP-expected6]
  imported/w3c/webdriver/tests/perform_actions/key_events.py::test_special_key_sends_keydown[SUBTRACT-expected16]
  imported/w3c/webdriver/tests/perform_actions/key_events.py::test_special_key_sends_keydown[PAGE_DOWN-expected30]
  imported/w3c/webdriver/tests/perform_actions/key_events.py::test_special_key_sends_keydown[ADD-expected40]
  imported/w3c/webdriver/tests/perform_actions/key_events.py::test_special_key_sends_keydown[NUMPAD5-expected61]
  imported/w3c/webdriver/tests/perform_actions/pointer_modifier_click.py::test_many_modifiers_click
  imported/w3c/webdriver/tests/minimize_window/minimize.py::test_fully_exit_fullscreen
  imported/w3c/webdriver/tests/get_element_tag_name/get.py::test_element_not_found
  imported/w3c/webdriver/tests/get_current_url/get.py::test_get_current_url_matches_location
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_accept[capabilities0-alert-None]
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_accept[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_accept[capabilities0-prompt-]
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_accept_and_notify[capabilities0-alert-None]
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_accept_and_notify[capabilities0-confirm-True]
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_accept_and_notify[capabilities0-prompt-]
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_dismiss[capabilities0-alert-None]
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_dismiss[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_dismiss[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_dismiss_and_notify[capabilities0-alert-None]
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_dismiss_and_notify[capabilities0-confirm-False]
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_dismiss_and_notify[capabilities0-prompt-None]
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_ignore[capabilities0-alert]
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_ignore[capabilities0-confirm]
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_ignore[capabilities0-prompt]
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_default[alert-None]
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_default[confirm-False]
  imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_default[prompt-None]
  imported/w3c/webdriver/tests/get_element_attribute/get.py::test_element_not_found

The patch updates WebDriver TestExpectations.json so there's no more expected to fail but passed tests.

[Carlos Garcia Campos]

Comment on attachment 395891 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=395891&action=review

> WebDriverTests/TestExpectations.json:336
> -        "expected": {"all": {"status": ["FAIL"], "bug": "webkit.org/b/188545"}},
> +        "expected": {"all": {"status": ["PASS"], "bug": "webkit.org/b/188545"}},

You don't need this, PASS is the expected result, simply add the subtest expectations for the failures.

> WebDriverTests/TestExpectations.json:434
> -        "expected": {"all": {"status": ["FAIL"], "bug": "webkit.org/b/182331"}},
> +        "expected": {"all": {"status": ["PASS"], "bug": "webkit.org/b/182331"}},

Ditto.

> WebDriverTests/TestExpectations.json:482
> +        "expected": {"all": {"status": ["PASS"]}},

Same here.

[Diego Pino]

Created attachment 395931 [details]
Patch

[EWS]

Committed r259791: <https://trac.webkit.org/changeset/259791>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 395931 [details].

[Radar WebKit Bug Importer]

<rdar://problem/61512043>