Bug 209646 – Regression(STP 103): 6 webdriver tests with new failures between STP 102 and 103

• Status: RESOLVED
• Resolution: MOVED
• Component: WebDriver

[Philip Jägenstedt]

In upgrading from STP 102 to 103 we see new failures in the webdriver tests, which is often bad news even though in this case it doesn't seem to lead to widespread failures in other tests.

The regressions can be seen here:
https://wpt.fyi/results/webdriver/tests?q=none%28status%3Amissing%29%20and%20seq%28status%3Apass%20status%3A%21pass%29&run_id=448800002&run_id=462410002

See https://github.com/web-platform-tests/wpt/pull/22475#issuecomment-604899967 for the details.

[Radar WebKit Bug Importer]

<rdar://problem/60990710>

[Blaze Burg]

There is some known flakiness in user prompt handling tests. I'll re-run these singly to see if these are failing in a uniquely bad way but it seems unlikely.

[Blaze Burg]

https://wpt.fyi/results/webdriver/tests/find_element/user_prompts.py?q=none%28status%3Amissing%29%20and%20seq%28status%3Apass%20status%3A%21pass%29&run_id=448800002&run_id=462410002



test_ignore[capabilities0-alert]	PASS	ERROR message: setup error
test_ignore[capabilities0-confirm]	PASS	ERROR message: setup error
test_ignore[capabilities0-prompt]	PASS	ERROR message: setup error
test_default[alert-None]	PASS	ERROR message: setup error
test_default[confirm-False]	PASS	ERROR message: setup error


I'm not sure what setup error is, but I'm assuming it's because the Safari is already paired with another session and is unable to create a new session. This is being tracked internally as a separate issue.

[Philip Jägenstedt]

Adding STP 105 results to the original comparison shows that there are only two remaining regressions:
https://wpt.fyi/results/webdriver/tests?q=seq%28status%3Apass%20status%3A%21pass%20status%3A%21pass%29&run_id=448800002&run_id=462410002&run_id=501500002

They are:
test_link_closes_window in element_click/navigate.py
test_default[alert-None] in get_active_element/user_prompts.py

[Brent Fulgham]

The fix for this issue was needed outside the WebKit project, therefore this is being resolved as 'Moved'.

This should now be fixed in shipping software.