Bug 217184 – [ iOS wk2 ews ]: 2 in-app-browser-privacy and 1 WPT are constant failures slowing down ews

• Status: RESOLVED
• Resolution: CONFIGURATION CHANGED
• Component: New Bugs

[Karl Rackler]

Description:
http/tests/in-app-browser-privacy/non-app-bound-domain-does-not-get-app-bound-session.html
http/tests/in-app-browser-privacy/top-frame-redirect-to-non-app-bound-domain-blocked.html
imported/w3c/web-platform-tests/xhr/event-timeout.any.html

These test are slowing down ews while awaiting completion of move EWS to new SDKs: iOS 14, tvOS 14, and watchOS 7 ( Bug 217145 ).

Queue History:
https://ews-build.webkit.org/#/builders/24

[Radar WebKit Bug Importer]

<rdar://problem/69845028>

[Karl Rackler]

I have marked these tests as failing while this issue is investigated on reproduction issues within open source and ews is impacted.
https://trac.webkit.org/changeset/267843/webkit

[Ryan Haddad]

http/tests/in-app-browser-privacy/non-app-bound-domain-does-not-get-app-bound-session.html is consistently failing on iOS 14 bots:

--- /Volumes/Data/slave/ios-simulator-14-release-tests-wk2/build/layout-test-results/http/tests/in-app-browser-privacy/non-app-bound-domain-does-not-get-app-bound-session-expected.txt
+++ /Volumes/Data/slave/ios-simulator-14-release-tests-wk2/build/layout-test-results/http/tests/in-app-browser-privacy/non-app-bound-domain-does-not-get-app-bound-session-actual.txt
@@ -3,8 +3,9 @@
 On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
 
 
-PASS Origin has non-app-bound session.
+FAIL Origin started with app-bound session.
 PASS successfullyParsed is true
+Some tests failed.
 
 TEST COMPLETE
 

http/tests/in-app-browser-privacy/top-frame-redirect-to-non-app-bound-domain-blocked.html is a flaky failure:

--- /Volumes/Data/slave/ios-simulator-14-release-tests-wk2/build/layout-test-results/http/tests/in-app-browser-privacy/top-frame-redirect-to-non-app-bound-domain-blocked-expected.txt
+++ /Volumes/Data/slave/ios-simulator-14-release-tests-wk2/build/layout-test-results/http/tests/in-app-browser-privacy/top-frame-redirect-to-non-app-bound-domain-blocked-actual.txt
@@ -1,10 +1 @@
-Test top frame redirects block non-app-bound domains
-
-On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
-
-
-PASS Redirect to non app-bound domain failed
-PASS successfullyParsed is true
-
-TEST COMPLETE
-
+FAILED: Should not have loaded

[Ryan Haddad]

imported/w3c/web-platform-tests/xhr/event-timeout.any.html is tracked with https://bugs.webkit.org/show_bug.cgi?id=216961

[Karl Rackler]

Reproduction attempt: 
[ Using 18A393, r267950 ]

Command: run-webkit-tests --ios-simulator --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1  http/tests/in-app-browser-privacy/non-app-bound-domain-does-not-get-app-bound-session.html

Result: 
Expected to fail, but passed: (1)
Unable to reproduce the open source dashboard (18A372) failure 

Another Reproduction attempt: 
[ Using 18A393, r267950 ]

Test: http/tests/in-app-browser-privacy/non-app-bound-domain-does-not-get-app-bound-session.html

Command: run-webkit-tests --ios-simulator --iterations 2000 --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 --debug-rwt-logging --no-retry --force --no-build -f  http/tests/in-app-browser-privacy/non-app-bound-domain-does-not-get-app-bound-session.html

Result: 
Results: 2000/2000 tests passed (100.0%)
Unable to reproduce the open source dashboard (18A372) failure

[Karl Rackler]

http/tests/in-app-browser-privacy/non-app-bound-domain-does-not-get-app-bound-session.html is tracked https://bugs.webkit.org/show_bug.cgi?id=217386

[Karl Rackler]

http/tests/in-app-browser-privacy/top-frame-redirect-to-non-app-bound-domain-blocked.html is tracked https://bugs.webkit.org/show_bug.cgi?id=217406