Bug 209646 - Regression(STP 103): 6 webdriver tests with new failures between STP 102 and 103
Summary: Regression(STP 103): 6 webdriver tests with new failures between STP 102 and 103
Status: RESOLVED MOVED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebDriver (show other bugs)
Version: Safari Technology Preview
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-03-27 02:25 PDT by Philip Jägenstedt
Modified: 2022-02-10 21:46 PST (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Philip Jägenstedt 2020-03-27 02:25:44 PDT
In upgrading from STP 102 to 103 we see new failures in the webdriver tests, which is often bad news even though in this case it doesn't seem to lead to widespread failures in other tests.

The regressions can be seen here:
https://wpt.fyi/results/webdriver/tests?q=none%28status%3Amissing%29%20and%20seq%28status%3Apass%20status%3A%21pass%29&run_id=448800002&run_id=462410002

See https://github.com/web-platform-tests/wpt/pull/22475#issuecomment-604899967 for the details.
Comment 1 Radar WebKit Bug Importer 2020-03-27 16:48:50 PDT
<rdar://problem/60990710>
Comment 2 BJ Burg 2020-03-30 16:02:37 PDT
There is some known flakiness in user prompt handling tests. I'll re-run these singly to see if these are failing in a uniquely bad way but it seems unlikely.
Comment 3 BJ Burg 2020-03-30 16:11:57 PDT
https://wpt.fyi/results/webdriver/tests/find_element/user_prompts.py?q=none%28status%3Amissing%29%20and%20seq%28status%3Apass%20status%3A%21pass%29&run_id=448800002&run_id=462410002



test_ignore[capabilities0-alert]	PASS	ERROR message: setup error
test_ignore[capabilities0-confirm]	PASS	ERROR message: setup error
test_ignore[capabilities0-prompt]	PASS	ERROR message: setup error
test_default[alert-None]	PASS	ERROR message: setup error
test_default[confirm-False]	PASS	ERROR message: setup error


I'm not sure what setup error is, but I'm assuming it's because the Safari is already paired with another session and is unable to create a new session. This is being tracked internally as a separate issue.
Comment 4 Philip Jägenstedt 2020-04-28 01:42:37 PDT
Adding STP 105 results to the original comparison shows that there are only two remaining regressions:
https://wpt.fyi/results/webdriver/tests?q=seq%28status%3Apass%20status%3A%21pass%20status%3A%21pass%29&run_id=448800002&run_id=462410002&run_id=501500002

They are:
test_link_closes_window in element_click/navigate.py
test_default[alert-None] in get_active_element/user_prompts.py
Comment 5 Brent Fulgham 2022-02-10 21:46:22 PST
The fix for this issue was needed outside the WebKit project, therefore this is being resolved as 'Moved'.

This should now be fixed in shipping software.