Bug 296255

Summary: [WebDriver] Selenium 4.34 tests teardown error due to reconnection errors on test failure
Product: WebKit Reporter: Lauro Moura <lmoura>
Component: WebDriverAssignee: Lauro Moura <lmoura>
Status: RESOLVED FIXED    
Severity: Normal CC: bburg, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on: 295405    
Bug Blocks:    

Lauro Moura
Reported 2025-07-20 16:31:46 PDT
In the bug295405 Selenium test update, the new bidi.storage tests are failing with what looks like reconnection errors: ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_get_cookie_by_name[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=40445): Max retries exceeded with url: /session/d6f7141a-b5a5-4e8c-813d-37470d6a0b77 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a0e9d7c0>: Failed to establish a new connection: [Errno 111] Conn... ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_get_cookie_in_default_user_context[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhos t', port=38865): Max retries exceeded with url: /session/14f040b5-45b1-4ac8-a5f9-c0a926afda49 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a064a810>: Failed to establish a new connection: [Errno 1 11] Conn... ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_get_cookie_in_a_user_context[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', po rt=56063): Max retries exceeded with url: /session/c42c3811-de0e-45f7-b51b-1f511b314dc0 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a0eed130>: Failed to establish a new connection: [Errno 111] Conn... ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_add_cookie[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=55731): Max ret ries exceeded with url: /session/263ae190-9cde-499a-beec-d1b13d39d955 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a0f24560>: Failed to establish a new connection: [Errno 111] Conn... ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_add_and_get_cookie[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=46527): Max retries exceeded with url: /session/269b3359-70d1-4258-b4d8-3da62db327f2 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a0efc470>: Failed to establish a new connection: [Errno 111] Conn... ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_get_all_cookies[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=33295): Ma x retries exceeded with url: /session/5f2d5ddb-c583-4408-8c32-dd8e6c18757f (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a0efcf80>: Failed to establish a new connection: [Errno 111] Conn... ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_delete_all_cookies[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=55909): Max retries exceeded with url: /session/e2de1f65-181f-465e-b607-4f7044bbaf5b (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a0efdb50>: Failed to establish a new connection: [Errno 111] Conn... ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_delete_cookie_with_name[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=58 891): Max retries exceeded with url: /session/10d95670-c128-463e-9adb-32dbb08339e9 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a02598e0>: Failed to establish a new connection: [Errno 111] Conn... ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_add_cookies_with_different_paths[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=51283): Max retries exceeded with url: /session/4c0afa55-349a-4164-b1c4-0a2fe483f36b (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a01a2450>: Failed to establish a new connection: [Errno 111 ] Conn... Not sure if the way the storage tests are run, inside a class, is messing up some fixture order or something like that.
Attachments
Add attachment proposed patch, testcase, etc.
Lauro Moura
Comment 1 2025-07-20 16:44:21 PDT
Just confirmed that this is not a storage-specific thing. In the `imported/selenium/py/test/selenium/webdriver/common/bidi_browsing_context_tests.py::test_activate_browsing_context[wpewebkit]` test we get the same issue: ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_browsing_context_tests.py::test_activate_browsing_context[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=43417): Max retries exceeded with url: /session/2dbd1aad-adca-4c11-9428-63076266640a (Caused by NewConnectionError('<urllib3.connection.HTTPConnection o... FAILED WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_browsing_context_tests.py::test_activate_browsing_context[wpewebkit] - assert False And verbose log from the failed assertion until the teardown: selenium.webdriver.remote.remote_connection: [DEBUG] POST http://localhost:43417/session/2dbd1aad-adca-4c11-9428-63076266640a/execute/sync {'script': 'return document.hasFocus();', 'args': []} urllib3.connectionpool: [DEBUG] http://localhost:43417 "POST /session/2dbd1aad-adca-4c11-9428-63076266640a/execute/sync HTTP/1.1" 200 15 selenium.webdriver.remote.remote_connection: [DEBUG] Remote response: status=200 | data={"value":false} | headers=HTTPHeaderDict({'Server': 'WebKitWebDriver', 'Date': 'Sun, 20 Jul 2025 23:41:51 GMT', 'Content-Type': 'application/json ; charset=utf-8', 'Cache-Control': 'no-cache', 'Content-Length': '15'}) selenium.webdriver.remote.remote_connection: [DEBUG] Finished Request FAILEDselenium.webdriver.remote.remote_connection: [DEBUG] DELETE http://localhost:43417/session/2dbd1aad-adca-4c11-9428-63076266640a {} urllib3.connectionpool: [DEBUG] http://localhost:43417 "DELETE /session/2dbd1aad-adca-4c11-9428-63076266640a HTTP/1.1" 200 14 selenium.webdriver.remote.remote_connection: [DEBUG] Remote response: status=200 | data={"value":null} | headers=HTTPHeaderDict({'Server': 'WebKitWebDriver', 'Date': 'Sun, 20 Jul 2025 23:41:51 GMT', 'Content-Type': 'application/json; charset=utf-8', 'Cache-Control': 'no-cache', 'Content-Length': '14'}) selenium.webdriver.remote.remote_connection: [DEBUG] Finished Request selenium.webdriver.remote.remote_connection: [DEBUG] DELETE http://localhost:43417/session/2dbd1aad-adca-4c11-9428-63076266640a {} urllib3.connectionpool: [DEBUG] Starting new HTTP connection (1): localhost:43417 urllib3.util.retry: [DEBUG] Incremented Retry for (url='/session/2dbd1aad-adca-4c11-9428-63076266640a'): Retry(total=2, connect=None, read=None, redirect=None, status=None) urllib3.connectionpool: [WARNING] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x718406060c50>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/2dbd1aad-adca-4c11-9428-63076266640a urllib3.connectionpool: [DEBUG] Starting new HTTP connection (2): localhost:43417 urllib3.util.retry: [DEBUG] Incremented Retry for (url='/session/2dbd1aad-adca-4c11-9428-63076266640a'): Retry(total=1, connect=None, read=None, redirect=None, status=None) urllib3.connectionpool: [WARNING] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x718406061160>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/2dbd1aad-adca-4c11-9428-63076266640a urllib3.connectionpool: [DEBUG] Starting new HTTP connection (3): localhost:43417 urllib3.util.retry: [DEBUG] Incremented Retry for (url='/session/2dbd1aad-adca-4c11-9428-63076266640a'): Retry(total=0, connect=None, read=None, redirect=None, status=None) urllib3.connectionpool: [WARNING] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x718406061790>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/2dbd1aad-adca-4c11-9428-63076266640a urllib3.connectionpool: [DEBUG] Starting new HTTP connection (4): localhost:43417 [2025-07-20 20:41:49] WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_browsing_context_tests.py::test_activate_browsing_context[wpewebkit] ERROR
Radar WebKit Bug Importer
Comment 2 2025-07-27 16:32:12 PDT
<rdar://problem/156868087>
Lauro Moura
Comment 3 2025-08-26 13:41:05 PDT
A bit more detail: - This manifests when a test case on a given file fails. If all tests are passing, no problem. - Interestingly, `xfail` reports the failure, but the teardown error doesn't seem to happen - The same selenium test suite works fine when invoked from bazel or manual pytest (from Selenium repository) (Also update the title as it's not BiDi-related)
Lauro Moura
Comment 4 2025-08-26 20:38:16 PDT
So, the issue is related to selenium's pytest fixtures trying to send the quit command after the original driver had already exited. I'll submit a patch upstream and likely do another import in this issue, as we already have selenium 4.35 out.
Lauro Moura
Comment 5 2025-09-14 20:11:38 PDT
Pull request: https://github.com/WebKit/WebKit/pull/50727
EWS
Comment 6 2025-09-25 21:50:09 PDT
Committed 300552@main (840ba04a5102): <https://commits.webkit.org/300552@main> Reviewed commits have been landed. Closing PR #50727 and removing active labels.
Note You need to log in before you can comment on or make changes to this bug.