Bug 296255 – [WebDriver] Selenium 4.34 tests teardown error due to reconnection errors on test failure

• Status: RESOLVED
• Resolution: FIXED
• Component: WebDriver

[Lauro Moura]

In the bug295405 Selenium test update, the new bidi.storage tests are failing with what looks like reconnection errors:

ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_get_cookie_by_name[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=40445):
 Max retries exceeded with url: /session/d6f7141a-b5a5-4e8c-813d-37470d6a0b77 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a0e9d7c0>: Failed to establish a new connection: [Errno 111] Conn...     
ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_get_cookie_in_default_user_context[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhos
t', port=38865): Max retries exceeded with url: /session/14f040b5-45b1-4ac8-a5f9-c0a926afda49 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a064a810>: Failed to establish a new connection: [Errno 1
11] Conn...                                                                                                                                                                                                                              
ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_get_cookie_in_a_user_context[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', po
rt=56063): Max retries exceeded with url: /session/c42c3811-de0e-45f7-b51b-1f511b314dc0 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a0eed130>: Failed to establish a new connection: [Errno 111] Conn...                                                                                                                                                                                                                                    
ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_add_cookie[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=55731): Max ret
ries exceeded with url: /session/263ae190-9cde-499a-beec-d1b13d39d955 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a0f24560>: Failed to establish a new connection: [Errno 111] Conn...             
ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_add_and_get_cookie[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=46527):
 Max retries exceeded with url: /session/269b3359-70d1-4258-b4d8-3da62db327f2 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a0efc470>: Failed to establish a new connection: [Errno 111] Conn...     ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_get_all_cookies[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=33295): Ma
x retries exceeded with url: /session/5f2d5ddb-c583-4408-8c32-dd8e6c18757f (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a0efcf80>: Failed to establish a new connection: [Errno 111] Conn...        
ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_delete_all_cookies[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=55909):
 Max retries exceeded with url: /session/e2de1f65-181f-465e-b607-4f7044bbaf5b (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a0efdb50>: Failed to establish a new connection: [Errno 111] Conn...     
ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_delete_cookie_with_name[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=58
891): Max retries exceeded with url: /session/10d95670-c128-463e-9adb-32dbb08339e9 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a02598e0>: Failed to establish a new connection: [Errno 111] Conn...
ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_storage_tests.py::TestBidiStorage::test_add_cookies_with_different_paths[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=51283): Max retries exceeded with url: /session/4c0afa55-349a-4164-b1c4-0a2fe483f36b (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7085a01a2450>: Failed to establish a new connection: [Errno 111
] Conn...     

Not sure if the way the storage tests are run, inside a class, is messing up some fixture order or something like that.

[Lauro Moura]

Just confirmed that this is not a storage-specific thing. In the `imported/selenium/py/test/selenium/webdriver/common/bidi_browsing_context_tests.py::test_activate_browsing_context[wpewebkit]` test we get the same issue:

ERROR WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_browsing_context_tests.py::test_activate_browsing_context[wpewebkit] - urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=43417): Max retries exceeded with url: /session/2dbd1aad-adca-4c11-9428-63076266640a (Caused by NewConnectionError('<urllib3.connection.HTTPConnection o...
FAILED WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_browsing_context_tests.py::test_activate_browsing_context[wpewebkit] - assert False

And verbose log from the failed assertion until the teardown:

selenium.webdriver.remote.remote_connection: [DEBUG] POST http://localhost:43417/session/2dbd1aad-adca-4c11-9428-63076266640a/execute/sync {'script': 'return document.hasFocus();', 'args': []}                                         
urllib3.connectionpool: [DEBUG] http://localhost:43417 "POST /session/2dbd1aad-adca-4c11-9428-63076266640a/execute/sync HTTP/1.1" 200 15                                                                                                 
selenium.webdriver.remote.remote_connection: [DEBUG] Remote response: status=200 | data={"value":false} | headers=HTTPHeaderDict({'Server': 'WebKitWebDriver', 'Date': 'Sun, 20 Jul 2025 23:41:51 GMT', 'Content-Type': 'application/json
; charset=utf-8', 'Cache-Control': 'no-cache', 'Content-Length': '15'})                                                                                                                                                                  selenium.webdriver.remote.remote_connection: [DEBUG] Finished Request                                                                                                                                                                    
FAILEDselenium.webdriver.remote.remote_connection: [DEBUG] DELETE http://localhost:43417/session/2dbd1aad-adca-4c11-9428-63076266640a {}                                                                                                 
urllib3.connectionpool: [DEBUG] http://localhost:43417 "DELETE /session/2dbd1aad-adca-4c11-9428-63076266640a HTTP/1.1" 200 14                                                                                                            
selenium.webdriver.remote.remote_connection: [DEBUG] Remote response: status=200 | data={"value":null} | headers=HTTPHeaderDict({'Server': 'WebKitWebDriver', 'Date': 'Sun, 20 Jul 2025 23:41:51 GMT', 'Content-Type': 'application/json;
 charset=utf-8', 'Cache-Control': 'no-cache', 'Content-Length': '14'})                                                                                                                                                                   selenium.webdriver.remote.remote_connection: [DEBUG] Finished Request                                                                                                                                                                    
selenium.webdriver.remote.remote_connection: [DEBUG] DELETE http://localhost:43417/session/2dbd1aad-adca-4c11-9428-63076266640a {}                                                                                                       
urllib3.connectionpool: [DEBUG] Starting new HTTP connection (1): localhost:43417                                                                                                                                                        
urllib3.util.retry: [DEBUG] Incremented Retry for (url='/session/2dbd1aad-adca-4c11-9428-63076266640a'): Retry(total=2, connect=None, read=None, redirect=None, status=None)                                                             
urllib3.connectionpool: [WARNING] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x718406060c50>: Failed to 
establish a new connection: [Errno 111] Connection refused')': /session/2dbd1aad-adca-4c11-9428-63076266640a                                                                                                                             urllib3.connectionpool: [DEBUG] Starting new HTTP connection (2): localhost:43417                                                                                                                                                        
urllib3.util.retry: [DEBUG] Incremented Retry for (url='/session/2dbd1aad-adca-4c11-9428-63076266640a'): Retry(total=1, connect=None, read=None, redirect=None, status=None)                                                             
urllib3.connectionpool: [WARNING] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x718406061160>: Failed to 
establish a new connection: [Errno 111] Connection refused')': /session/2dbd1aad-adca-4c11-9428-63076266640a                                                                                                                             
urllib3.connectionpool: [DEBUG] Starting new HTTP connection (3): localhost:43417                                                                                                                                                        
urllib3.util.retry: [DEBUG] Incremented Retry for (url='/session/2dbd1aad-adca-4c11-9428-63076266640a'): Retry(total=0, connect=None, read=None, redirect=None, status=None)                                                             
urllib3.connectionpool: [WARNING] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x718406061790>: Failed to 
establish a new connection: [Errno 111] Connection refused')': /session/2dbd1aad-adca-4c11-9428-63076266640a                                                                                                                             
urllib3.connectionpool: [DEBUG] Starting new HTTP connection (4): localhost:43417                                                                                                                                                        
                                                                                                                                                                                                                                         
[2025-07-20 20:41:49] WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_browsing_context_tests.py::test_activate_browsing_context[wpewebkit] ERROR

[Radar WebKit Bug Importer]

<rdar://problem/156868087>

[Lauro Moura]

A bit more detail:

- This manifests when a test case on a given file fails. If all tests are passing, no problem.
- Interestingly, `xfail` reports the failure, but the teardown error doesn't seem to happen
- The same selenium test suite works fine when invoked from bazel or manual pytest (from Selenium repository)

(Also update the title as it's not BiDi-related)

[Lauro Moura]

So, the issue is related to selenium's pytest fixtures trying to send the quit command after the original driver had already exited.

I'll submit a patch upstream and likely do another import in this issue, as we already have selenium 4.35 out.

[Lauro Moura]

Pull request: https://github.com/WebKit/WebKit/pull/50727

[EWS]

Committed 300552@main (840ba04a5102): <https://commits.webkit.org/300552@main>

Reviewed commits have been landed. Closing PR #50727 and removing active labels.