Many WebDriver tests are failing: Unexpected failures (11) imported/selenium/py/test/selenium/webdriver/common/alerts_tests.py::testShouldHandleAlertOnPageBeforeUnload[WebKitGTK] imported/selenium/py/test/selenium/webdriver/common/page_load_timeout_tests.py::testClickShouldTimeout[WebKitGTK] imported/selenium/py/test/selenium/webdriver/common/page_load_timeout_tests.py::testClickShouldTimeout[WebKitGTK] imported/selenium/py/test/selenium/webdriver/common/window_tests.py::test_should_fullscreen_the_current_window[WebKitGTK] imported/w3c/webdriver/tests/close_window/user_prompts.py::test_handle_prompt_accept imported/w3c/webdriver/tests/close_window/close.py::test_close_last_browsing_context imported/w3c/webdriver/tests/set_window_rect/set.py::test_fully_exit_fullscreen imported/w3c/webdriver/tests/maximize_window/maximize.py::test_fully_exit_fullscreen imported/w3c/webdriver/tests/new_session/response.py::test_resp_capabilites imported/w3c/webdriver/tests/new_session/response.py::test_resp_data imported/w3c/webdriver/tests/minimize_window/minimize.py::test_fully_exit_fullscreen Unexpected timeouts (1) imported/w3c/webdriver/tests/actions/special_keys.py::test_webdriver_special_key_sends_keydown[F5-expected55] Of those, page_load_timeout_tests.py, set.py, maximize.py, and minimize.py are all flaky. I think the rest are consistent failures. This is too many failures for me to report separate bugs for each one. I'll use separate comments. Feel free to split failures into separate bugs as desired.
______________ testShouldHandleAlertOnPageBeforeUnload[WebKitGTK] ______________ driver = <selenium.webdriver.webkitgtk.webdriver.WebDriver (session="768f3a45-1b30-4b4e-9b6e-ea1ad8c3c301")> pages = <conftest.Pages object at 0x7f6061968a10> def testShouldHandleAlertOnPageBeforeUnload(driver, pages): pages.load("pageWithOnBeforeUnloadMessage.html") element = driver.find_element(By.ID, "navigate") element.click() > alert = _waitForAlert(driver) driver = <selenium.webdriver.webkitgtk.webdriver.WebDriver (session="768f3a45-1b30-4b4e-9b6e-ea1ad8c3c301")> element = <selenium.webdriver.remote.webelement.WebElement (session="768f3a45-1b30-4b4e-9b6e-ea1ad8c3c301", element="node-A7AFE675-0BF1-4957-8B3B-BFB8ACE80CE1")> pages = <conftest.Pages object at 0x7f6061968a10> WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/alerts_tests.py:249: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/alerts_tests.py:318: in _waitForAlert return WebDriverWait(driver, 3).until(EC.alert_is_present()) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <selenium.webdriver.support.wait.WebDriverWait (session="768f3a45-1b30-4b4e-9b6e-ea1ad8c3c301")> method = <selenium.webdriver.support.expected_conditions.alert_is_present object at 0x7f6061968210> message = '' def until(self, method, message=''): """Calls the method provided with the driver as an argument until the \ return value is not False.""" screen = None stacktrace = None end_time = time.time() + self._timeout while True: try: value = method(self._driver) if value: return value except self._ignored_exceptions as exc: screen = getattr(exc, 'screen', None) stacktrace = getattr(exc, 'stacktrace', None) time.sleep(self._poll) if time.time() > end_time: break > raise TimeoutException(message, screen, stacktrace) E TimeoutException: Message: end_time = 1532715605.473464 message = '' method = <selenium.webdriver.support.expected_conditions.alert_is_present object at 0x7f6061968210> screen = None self = <selenium.webdriver.support.wait.WebDriverWait (session="768f3a45-1b30-4b4e-9b6e-ea1ad8c3c301")> stacktrace = None value = False WebDriverTests/imported/selenium/py/selenium/webdriver/support/wait.py:80: TimeoutException
____________ ERROR at teardown of testClickShouldTimeout[WebKitGTK] ____________ driver = <selenium.webdriver.webkitgtk.webdriver.WebDriver (session="165d5c3a-2682-4b25-9434-5672d2903e95")> @pytest.fixture(autouse=True) def reset_timeouts(driver): yield > driver.set_page_load_timeout(300) driver = <selenium.webdriver.webkitgtk.webdriver.WebDriver (session="165d5c3a-2682-4b25-9434-5672d2903e95")> WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/page_load_timeout_tests.py:26: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ WebDriverTests/imported/selenium/py/selenium/webdriver/remote/webdriver.py:926: in set_page_load_timeout 'pageLoad': int(float(time_to_wait) * 1000)}) WebDriverTests/imported/selenium/py/selenium/webdriver/remote/webdriver.py:310: in execute response = self.command_executor.execute(driver_command, params) WebDriverTests/imported/selenium/py/selenium/webdriver/remote/remote_connection.py:466: in execute return self._request(command_info[0], url, body=data) WebDriverTests/imported/selenium/py/selenium/webdriver/remote/remote_connection.py:528: in _request resp = opener.open(request, timeout=self._timeout) /usr/lib/python2.7/urllib2.py:429: in open response = self._open(req, data) /usr/lib/python2.7/urllib2.py:447: in _open '_open', req) /usr/lib/python2.7/urllib2.py:407: in _call_chain result = func(*args) /usr/lib/python2.7/urllib2.py:1228: in http_open return self.do_open(httplib.HTTPConnection, req) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib2.HTTPHandler instance at 0x7f6062a34878> http_class = <class httplib.HTTPConnection at 0x7f60682bb050> req = <selenium.webdriver.remote.remote_connection.Request instance at 0x7f6062a34e60> http_conn_args = {}, host = '127.0.0.1:52629' h = <httplib.HTTPConnection instance at 0x7f6062a348c0> err = error(111, 'Connection refused') def do_open(self, http_class, req, **http_conn_args): """Return an addinfourl object for the request, using http_class. http_class must implement the HTTPConnection API from httplib. The addinfourl return value is a file-like object. It also has methods and attributes including: - info(): return a mimetools.Message object for the headers - geturl(): return the original request URL - code: HTTP status code """ host = req.get_host() if not host: raise URLError('no host given') # will parse host:port h = http_class(host, timeout=req.timeout, **http_conn_args) h.set_debuglevel(self._debuglevel) headers = dict(req.unredirected_hdrs) headers.update(dict((k, v) for k, v in req.headers.items() if k not in headers)) # We want to make an HTTP/1.1 request, but the addinfourl # class isn't prepared to deal with a persistent connection. # It will try to read all remaining data from the socket, # which will block while the server waits for the next request. # So make sure the connection gets closed after the (only) # request. headers["Connection"] = "close" headers = dict( (name.title(), val) for name, val in headers.items()) if req._tunnel_host: tunnel_headers = {} proxy_auth_hdr = "Proxy-Authorization" if proxy_auth_hdr in headers: tunnel_headers[proxy_auth_hdr] = headers[proxy_auth_hdr] # Proxy-Authorization should not be sent to origin # server. del headers[proxy_auth_hdr] h.set_tunnel(req._tunnel_host, headers=tunnel_headers) try: h.request(req.get_method(), req.get_selector(), req.data, headers) except socket.error, err: # XXX what error? h.close() > raise URLError(err) E URLError: <urlopen error [Errno 111] Connection refused> err = error(111, 'Connection refused') h = <httplib.HTTPConnection instance at 0x7f6062a348c0> headers = {'Accept': 'application/json', 'Connection': 'close', 'Content-Length': '20', 'Content-Type': 'application/json;charset=UTF-8', ...} host = '127.0.0.1:52629' http_class = <class httplib.HTTPConnection at 0x7f60682bb050> http_conn_args = {} req = <selenium.webdriver.remote.remote_connection.Request instance at 0x7f6062a34e60> self = <urllib2.HTTPHandler instance at 0x7f6062a34878> /usr/lib/python2.7/urllib2.py:1198: URLError
_____________ test_should_fullscreen_the_current_window[WebKitGTK] _____________ driver = <selenium.webdriver.webkitgtk.webdriver.WebDriver (session="e15545c7-7d93-476c-a5c2-d1c8aaab5d35")> @pytest.mark.xfail_chrome(raises=WebDriverException, reason='Fullscreen command not implemented') @pytest.mark.xfail_firefox(raises=WebDriverException, reason='Fullscreen command not implemented') @pytest.mark.xfail_remote(raises=WebDriverException, reason='Fullscreen command not implemented') @pytest.mark.xfail_safari(raises=WebDriverException, reason='Fullscreen command not implemented') @pytest.mark.skipif(os.environ.get('CI') == 'true', reason='Fullscreen command causes Travis to hang') def test_should_fullscreen_the_current_window(driver): start_width = driver.execute_script('return window.innerWidth;') start_height = driver.execute_script('return window.innerHeight;') driver.fullscreen_window() > WebDriverWait(driver, 2).until(lambda d: driver.execute_script('return window.innerWidth;') > start_width) driver = <selenium.webdriver.webkitgtk.webdriver.WebDriver (session="e15545c7-7d93-476c-a5c2-d1c8aaab5d35")> start_height = 716 start_width = 1014 WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/window_tests.py:140: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <selenium.webdriver.support.wait.WebDriverWait (session="e15545c7-7d93-476c-a5c2-d1c8aaab5d35")> method = <function <lambda> at 0x7f6062df12a8>, message = '' def until(self, method, message=''): """Calls the method provided with the driver as an argument until the \ return value is not False.""" screen = None stacktrace = None end_time = time.time() + self._timeout while True: try: value = method(self._driver) if value: return value except self._ignored_exceptions as exc: screen = getattr(exc, 'screen', None) stacktrace = getattr(exc, 'stacktrace', None) time.sleep(self._poll) if time.time() > end_time: break > raise TimeoutException(message, screen, stacktrace) E TimeoutException: Message: end_time = 1532716091.136181 message = '' method = <function <lambda> at 0x7f6062df12a8> screen = None self = <selenium.webdriver.support.wait.WebDriverWait (session="e15545c7-7d93-476c-a5c2-d1c8aaab5d35")> stacktrace = None value = False WebDriverTests/imported/selenium/py/selenium/webdriver/support/wait.py:80: TimeoutException
__________________________ test_handle_prompt_accept ___________________________ new_session = <function create_session at 0x7f60619b5d70> add_browser_capabilites = <function update_capabilities at 0x7f60619b5b18> def test_handle_prompt_accept(new_session, add_browser_capabilites): _, session = new_session({"capabilities": { "alwaysMatch": add_browser_capabilites({"unhandledPromptBehavior": "accept"})}}) session.window_handle = create_window(session)() session.url = inline("<title>WD doc title</title>") create_dialog(session)("alert", text="dismiss #1", result_var="dismiss1") response = close(session) assert response.status == 200 > assert_dialog_handled(session, "dismiss #1") _ = {'capabilities': {'browserName': 'MiniBrowser', 'browserVersion': '2.21.5', 'pageLoadStrategy': 'normal', 'platformName': 'linux', ...}, 'sessionId': '0ab3c417-b701-462d-bbbf-5590b1e34149'} add_browser_capabilites = <function update_capabilities at 0x7f60619b5b18> new_session = <function create_session at 0x7f60619b5d70> response = <Responsetatus=200 body={"value": ["page-3DC2B05B-A549-41AC-B31B-E91082BCBA74"]}> session = <Session 0ab3c417-b701-462d-bbbf-5590b1e34149> WebDriverTests/imported/w3c/webdriver/tests/close_window/user_prompts.py:35: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ session = <Session 0ab3c417-b701-462d-bbbf-5590b1e34149> expected_text = 'dismiss #1' def assert_dialog_handled(session, expected_text): result = session.transport.send("GET", "session/%s/alert/text" % session.session_id) # If there were any existing dialogs prior to the creation of this # fixture's dialog, then the "Get Alert Text" command will return # successfully. In that case, the text must be different than that # of this fixture's dialog. try: assert_error(result, "no such alert") except: > assert (result.status == 200 and result.body["value"] != expected_text), ( "Dialog with text '%s' was not handled." % expected_text) E AssertionError: Dialog with text 'dismiss #1' was not handled. E assert (404 == 200) E + where 404 = <Response status=404 error=<NoSuchWindowException http_status=404>>.status expected_text = 'dismiss #1' result = <Response status=404 error=<NoSuchWindowException http_status=404>> session = <Session 0ab3c417-b701-462d-bbbf-5590b1e34149> WebDriverTests/imported/w3c/webdriver/tests/support/asserts.py:101: AssertionError
WebDriverTests/imported/w3c/webdriver/tests/close_window/close.py::test_no_browsing_context PASSED WebDriverTests/imported/w3c/webdriver/tests/close_window/close.py::test_close_browsing_context PASSED WebDriverTests/imported/w3c/webdriver/tests/close_window/close.py::test_close_last_browsing_context PASSEDIgnored exception session not created (500): Failed to connect to browser: Failed to execute child process “/usr/local/libexec/webkit2gtk-4.0/MiniBrowser” (No such file or directory) Ignored exception session not created (500): Failed to connect to browser: Failed to execute child process “/usr/local/libexec/webkit2gtk-4.0/MiniBrowser” (No such file or directory) Ignored exception session not created (500): Failed to connect to browser: Failed to execute child process “/usr/local/libexec/webkit2gtk-4.0/MiniBrowser” (No such file or directory) Ignored exception session not created (500): Failed to connect to browser: Failed to execute child process “/usr/local/libexec/webkit2gtk-4.0/MiniBrowser” (No such file or directory) WebDriverTests/imported/w3c/webdriver/tests/close_window/close.py::test_close_last_browsing_context ERROR ==================================== ERRORS ==================================== ____________ ERROR at teardown of test_close_last_browsing_context _____________ > request.addfinalizer(lambda: _switch_to_top_level_browsing_context(_current_session)) WebDriverTests/imported/w3c/webdriver/tests/support/fixtures.py:166: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ WebDriverTests/imported/w3c/webdriver/tests/support/fixtures.py:21: in inner return f(*args, **kwargs) WebDriverTests/imported/w3c/webdriver/tests/support/fixtures.py:85: in _switch_to_top_level_browsing_context session.switch_frame(None) WebDriverTests/imported/w3c/tools/webdriver/webdriver/client.py:535: in switch_frame return self.send_session_command("POST", url, body) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <Session (disconnected)>, method = 'POST', uri = 'frame' body = {'id': None} def send_session_command(self, method, uri, body=None): """ Send a command to an established session and validate its success. :param method: HTTP method to use in request. :param url: "Command part" of the HTTP request URL, e.g. `window/rect`. :param body: Optional body of the HTTP request. Must be JSON serialisable. :return: `None` if the HTTP response body was empty, otherwise the result of parsing the body as JSON. :raises error.SessionNotCreatedException: If there is no active session. :raises error.WebDriverException: If the remote end returns an error. """ if self.session_id is None: > raise error.SessionNotCreatedException() E TypeError: __init__() takes at least 2 arguments (1 given) body = {'id': None} method = 'POST' self = <Session (disconnected)> uri = 'frame'
__________________________ test_fully_exit_fullscreen __________________________ session = <Session 32832ce0-1f20-41d0-a140-9809e8c899c3> def test_fully_exit_fullscreen(session): """ 10. Fully exit fullscreen. [...] To fully exit fullscreen a document document, run these steps: 1. If document's fullscreen element is null, terminate these steps. 2. Unfullscreen elements whose fullscreen flag is set, within document's top layer, except for document's fullscreen element. 3. Exit fullscreen document. """ > session.window.fullscreen() session = <Session 32832ce0-1f20-41d0-a140-9809e8c899c3> WebDriverTests/imported/w3c/webdriver/tests/set_window_rect/set.py:192: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ WebDriverTests/imported/w3c/tools/webdriver/webdriver/client.py:21: in inner return func(self, *args, **kwargs) WebDriverTests/imported/w3c/tools/webdriver/webdriver/client.py:285: in fullscreen return self.session.send_session_command("POST", "window/fullscreen") WebDriverTests/imported/w3c/tools/webdriver/webdriver/client.py:479: in send_session_command return self.send_command(method, url, body) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <Session 32832ce0-1f20-41d0-a140-9809e8c899c3>, method = 'POST' url = 'session/32832ce0-1f20-41d0-a140-9809e8c899c3/window/fullscreen' body = None def send_command(self, method, url, body=None): """ Send a command to the remote end and validate its success. :param method: HTTP method to use in request. :param uri: "Command part" of the HTTP request URL, e.g. `window/rect`. :param body: Optional body of the HTTP request. :return: `None` if the HTTP response body was empty, otherwise the `value` field returned after parsing the response body as JSON. :raises ValueError: If the response body does not contain a `value` key. :raises error.WebDriverException: If the remote end returns an error. """ response = self.transport.send( method, url, body, encoder=protocol.Encoder, decoder=protocol.Decoder, session=self) if response.status != 200: > raise error.from_response(response) E ScriptTimeoutException: script timeout (408): body = None method = 'POST' response = <Response status=408 error=<ScriptTimeoutException http_status=408>> self = <Session 32832ce0-1f20-41d0-a140-9809e8c899c3> url = 'session/32832ce0-1f20-41d0-a140-9809e8c899c3/window/fullscreen' WebDriverTests/imported/w3c/tools/webdriver/webdriver/client.py:438: ScriptTimeoutException
____________________________ test_resp_capabilites _____________________________ new_session = <function create_session at 0x7f606192f230> add_browser_capabilites = <function update_capabilities at 0x7f606192f1b8> def test_resp_capabilites(new_session, add_browser_capabilites): resp, _ = new_session({"capabilities": {"alwaysMatch": add_browser_capabilites({})}}) assert isinstance(resp["sessionId"], unicode) assert isinstance(resp["capabilities"], dict) > assert {"browserName", "browserVersion", "platformName", "acceptInsecureCerts", "setWindowRect", "timeouts", "proxy", "pageLoadStrategy"}.issubset( set(resp["capabilities"].keys())) E AssertionError: assert False E + where False = <built-in method issubset of set object at 0x7f6062c7e5c0>(set(['browserName', 'browserVersion', 'pageLoadStrategy', 'platformName', 'proxy', 'setWindowRect', ...])) E + where <built-in method issubset of set object at 0x7f6062c7e5c0> = set(['acceptInsecureCerts', 'browserName', 'browserVersion', 'pageLoadStrategy', 'platformName', 'proxy', ...]).issubset E + and set(['browserName', 'browserVersion', 'pageLoadStrategy', 'platformName', 'proxy', 'setWindowRect', ...]) = set(['browserVersion', 'timeouts', 'browserName', 'setWindowRect', 'proxy', 'pageLoadStrategy', ...]) E + where ['browserVersion', 'timeouts', 'browserName', 'setWindowRect', 'proxy', 'pageLoadStrategy', ...] = <built-in method keys of dict object at 0x7f6061957c58>() E + where <built-in method keys of dict object at 0x7f6061957c58> = {'browserName': 'MiniBrowser', 'browserVersion': '2.21.5', 'pageLoadStrategy': 'normal', 'platformName': 'linux', ...}.keys _ = <Session 2ba6e599-c519-436c-b896-4804f8947ac7> add_browser_capabilites = <function update_capabilities at 0x7f606192f1b8> new_session = <function create_session at 0x7f606192f230> resp = {'capabilities': {'browserName': 'MiniBrowser', 'browserVersion': '2.21.5', 'pageLoadStrategy': 'normal', 'platformName': 'linux', ...}, 'sessionId': '2ba6e599-c519-436c-b896-4804f8947ac7'} WebDriverTests/imported/w3c/webdriver/tests/new_session/response.py:15: AssertionError ________________________________ test_resp_data ________________________________ new_session = <function create_session at 0x7f60630dcde8> add_browser_capabilites = <function update_capabilities at 0x7f6062ddd2a8> platform_name = 'linux' def test_resp_data(new_session, add_browser_capabilites, platform_name): resp, _ = new_session({"capabilities": {"alwaysMatch": add_browser_capabilites({})}}) assert isinstance(resp["capabilities"]["browserName"], unicode) assert isinstance(resp["capabilities"]["browserVersion"], unicode) if platform_name: assert resp["capabilities"]["platformName"] == platform_name else: assert "platformName" in resp["capabilities"] > assert resp["capabilities"]["acceptInsecureCerts"] is False E KeyError: 'acceptInsecureCerts' _ = <Session a4e36348-81ad-49d4-897c-246fc076d822> add_browser_capabilites = <function update_capabilities at 0x7f6062ddd2a8> new_session = <function create_session at 0x7f60630dcde8> platform_name = 'linux' resp = {'capabilities': {'browserName': 'MiniBrowser', 'browserVersion': '2.21.5', 'pageLoadStrategy': 'normal', 'platformName': 'linux', ...}, 'sessionId': 'a4e36348-81ad-49d4-897c-246fc076d822'} WebDriverTests/imported/w3c/webdriver/tests/new_session/response.py:35: KeyError
Mmmm, the WebDriver expectations are not cooperating. I'm gcurrently sitting at: Expected to fail, but passed (3) imported/w3c/webdriver/tests/close_window/close.py::test_close_last_browsing_context imported/w3c/webdriver/tests/maximize_window/maximize.py::test_fully_exit_fullscreen imported/w3c/webdriver/tests/minimize_window/minimize.py::test_fully_exit_fullscreen All of the above have flaky expectations, just like for API tests. Perhaps the test runner is not able to handle multiple expectations? The same syntax works for API tests. Unexpected failures (4) imported/selenium/py/test/selenium/webdriver/common/alerts_tests.py::testShouldHandleAlertOnPageBeforeUnload[WebKitGTK] imported/selenium/py/test/selenium/webdriver/common/page_load_timeout_tests.py::testClickShouldTimeout[WebKitGTK] imported/selenium/py/test/selenium/webdriver/common/page_load_timeout_tests.py::testClickShouldTimeout[WebKitGTK] imported/selenium/py/test/selenium/webdriver/common/window_tests.py::test_should_fullscreen_the_current_window[WebKitGTK] Ditto, except for that last test, which has just a FAIL expectation. I'll try removing the "[WebKitGTK]" from the end of the expectations and see if it makes any difference. Unexpected timeouts (1) imported/w3c/webdriver/tests/actions/special_keys.py::test_webdriver_special_key_sends_keydown[F5-expected55] This one is skipped, and has been for ages, so at least this one should not be my fault. :)
I think the test runner can't handle flakiness yet. So I'm skipping the following tests: imported/selenium/py/test/selenium/webdriver/common/page_load_timeout_tests.py::testClickShouldTimeout imported/w3c/webdriver/tests/set_window_rect/set.py::test_fully_exit_fullscreen imported/w3c/webdriver/tests/maximize_window/maximize.py::test_fully_exit_fullscreen imported/w3c/webdriver/tests/maximize_window/minimize.py::test_fully_exit_fullscreen
imported/w3c/webdriver/tests/close_window/close.py::test_close_last_browsing_context is now passing.
These tests are all flaky: imported/w3c/webdriver/tests/element_clear/clear.py::test_input_file imported/w3c/webdriver/tests/element_clear/clear.py::test_input_file_multiple imported/w3c/webdriver/tests/element_clear/clear.py::test_resettable_element_does_not_satisfy_validation_constraints[number-foo] imported/w3c/webdriver/tests/element_clear/clear.py::test_resettable_element_does_not_satisfy_validation_constraints[range-foo] imported/w3c/webdriver/tests/element_clear/clear.py::test_resettable_element_does_not_satisfy_validation_constraints[email-foo] imported/w3c/webdriver/tests/element_clear/clear.py::test_resettable_element_does_not_satisfy_validation_constraints[url-foo] imported/w3c/webdriver/tests/element_clear/clear.py::test_resettable_element_does_not_satisfy_validation_constraints[color-foo] imported/w3c/webdriver/tests/element_clear/clear.py::test_resettable_element_does_not_satisfy_validation_constraints[date-foo] imported/w3c/webdriver/tests/element_clear/clear.py::test_resettable_element_does_not_satisfy_validation_constraints[datetime-foo] imported/w3c/webdriver/tests/element_clear/clear.py::test_resettable_element_does_not_satisfy_validation_constraints[datetime-local-foo] imported/w3c/webdriver/tests/element_clear/clear.py::test_resettable_element_does_not_satisfy_validation_constraints[time-foo] imported/w3c/webdriver/tests/element_clear/clear.py::test_resettable_element_does_not_satisfy_validation_constraints[month-foo] imported/w3c/webdriver/tests/element_clear/clear.py::test_resettable_element_does_not_satisfy_validation_constraints[week-foo] Since the test runner can't handle multiple expectations for the same test, I will skip them.
(In reply to Michael Catanzaro from comment #11) > These tests are all flaky: > > imported/w3c/webdriver/tests/element_clear/clear.py::test_input_file > > imported/w3c/webdriver/tests/element_clear/clear.py::test_input_file_multiple > > imported/w3c/webdriver/tests/element_clear/clear.py:: > test_resettable_element_does_not_satisfy_validation_constraints[number-foo] > > imported/w3c/webdriver/tests/element_clear/clear.py:: > test_resettable_element_does_not_satisfy_validation_constraints[range-foo] > > imported/w3c/webdriver/tests/element_clear/clear.py:: > test_resettable_element_does_not_satisfy_validation_constraints[email-foo] > > imported/w3c/webdriver/tests/element_clear/clear.py:: > test_resettable_element_does_not_satisfy_validation_constraints[url-foo] > > imported/w3c/webdriver/tests/element_clear/clear.py:: > test_resettable_element_does_not_satisfy_validation_constraints[color-foo] > > imported/w3c/webdriver/tests/element_clear/clear.py:: > test_resettable_element_does_not_satisfy_validation_constraints[date-foo] > > imported/w3c/webdriver/tests/element_clear/clear.py:: > test_resettable_element_does_not_satisfy_validation_constraints[datetime-foo] > > imported/w3c/webdriver/tests/element_clear/clear.py:: > test_resettable_element_does_not_satisfy_validation_constraints[datetime- > local-foo] > > imported/w3c/webdriver/tests/element_clear/clear.py:: > test_resettable_element_does_not_satisfy_validation_constraints[time-foo] > > imported/w3c/webdriver/tests/element_clear/clear.py:: > test_resettable_element_does_not_satisfy_validation_constraints[month-foo] > > imported/w3c/webdriver/tests/element_clear/clear.py:: > test_resettable_element_does_not_satisfy_validation_constraints[week-foo] > > > Since the test runner can't handle multiple expectations for the same test, > I will skip them. Sorry, let's track these in bug #180404, not here.
New failures: imported/w3c/webdriver/tests/element_send_keys/events.py::test_not_blurred[input] imported/w3c/webdriver/tests/element_send_keys/events.py::test_not_blurred[textarea] imported/w3c/webdriver/tests/element_click/navigate.py::test_link_unload_event imported/w3c/webdriver/tests/actions/control_click.py::test_control_click[\ue009-ctrlKey] imported/w3c/webdriver/tests/actions/control_click.py::test_control_click[\ue051-ctrlKey] imported/w3c/webdriver/tests/actions/control_click.py::test_release_control_click imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_accept[capabilities0-prompt-] imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_dismiss[capabilities0-prompt-None] New flakes (skipped): imported/w3c/webdriver/tests/fullscreen_window/user_prompts.py::test_accept[capabilities0-alert-None] imported/w3c/webdriver/tests/fullscreen_window/user_prompts.py::test_accept[capabilities0-confirm-True] imported/w3c/webdriver/tests/fullscreen_window/user_prompts.py::test_accept[capabilities0-prompt-] imported/w3c/webdriver/tests/fullscreen_window/user_prompts.py::test_dismiss[capabilities0-alert-None] imported/w3c/webdriver/tests/fullscreen_window/user_prompts.py::test_dismiss[capabilities0-confirm-False] imported/w3c/webdriver/tests/fullscreen_window/user_prompts.py::test_dismiss[capabilities0-prompt-None] imported/w3c/webdriver/tests/fullscreen_window/fullscreen.py::test_fullscreen imported/w3c/webdriver/tests/fullscreen_window/fullscreen.py::test_payload New timeouts: imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_accept[capabilities0-alert-None] imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_accept[capabilities0-confirm-True] imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_accept[capabilities0-prompt-] imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_dismiss[capabilities0-alert-None] imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_dismiss[capabilities0-confirm-False] imported/w3c/webdriver/tests/element_send_keys/user_prompts.py::test_dismiss[capabilities0-prompt-None] imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_accept[capabilities0-confirm-True] imported/w3c/webdriver/tests/is_element_selected/user_prompts.py::test_dismiss[capabilities0-confirm-False] New flaky timeout: imported/w3c/webdriver/tests/element_clear/clear.py::test_input_file
Actually these ones are flaky too. I'm just going to skip all the user_prompts.py tests. (In reply to Michael Catanzaro from comment #13) > imported/w3c/webdriver/tests/is_element_selected/user_prompts.py:: > test_accept[capabilities0-prompt-] > > imported/w3c/webdriver/tests/is_element_selected/user_prompts.py:: > test_dismiss[capabilities0-prompt-None]