Bug 230011 – [ iOS EWS ] imported/w3c/web-platform-tests/html/cross-origin-opener-policy/resource-popup.https.html is flaky

• Status: NEW
• Component: DOM

imported/w3c/web-platform-tests/html/cross-origin-opener-policy/resource-popup.https.html

Is flaky crashing on iOS-14-Simulator-WK2-Tests-EWS

The test started showing up as a flaky test at https://ews-build.webkit.org/#/builders/51/builds/20556

The test seems to have been a flaky failure/ crash in the open source directory: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fcross-origin-opener-policy%2Fresource-popup.https.html

Crash log:

Failed to find media engine.
com.apple.WebKit.WebContent.Development terminated (pid 22070) because the process crashed

[Radar WebKit Bug Importer]

<rdar://problem/82829778>

Updated test expectations to speed up EWS https://trac.webkit.org/changeset/282095/webkit

[Alexey Proskuryakov]

This is a test for COOP.

The test is flaky failing on BigSur wk2 Release as well.

Diff:

--- /Volumes/Data/worker/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/resource-popup.https-expected.txt
+++ /Volumes/Data/worker/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/resource-popup.https-actual.txt
@@ -2,6 +2,8 @@
 
 The HTTP `Refresh` header is used to ensure the child eventually closes itself (since proper observance of COOP will prevent the parent from closing the child in some cases).
 
+
+Harness Error (TIMEOUT), message = null
 
 PASS /common/dummy.xml - parent COOP: ""; child COOP: "same-origin"
 PASS /common/dummy.xml - parent COOP: "same-origin"; child COOP: ""
@@ -9,7 +11,7 @@
 PASS /images/red.png - parent COOP: ""; child COOP: "same-origin"
 PASS /images/red.png - parent COOP: "same-origin"; child COOP: ""
 PASS /images/red.png - parent COOP: "same-origin"; child COOP: "same-origin"
-PASS /common/text-plain.txt - parent COOP: ""; child COOP: "same-origin"
+TIMEOUT /common/text-plain.txt - parent COOP: ""; child COOP: "same-origin" Test timed out
 PASS /common/text-plain.txt - parent COOP: "same-origin"; child COOP: ""
 PASS /common/text-plain.txt - parent COOP: "same-origin"; child COOP: "same-origin"
 PASS /media/2x2-green.mp4 - parent COOP: ""; child COOP: "same-origin"

From the test name and diff, this might be related to Bug 228642.

I was not able to reproduce the failure locally on iOS 14 simulator using run-webkit-tests --force --iterations 100 --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 imported/w3c/web-platform-tests/html/cross-origin-opener-policy/resource-popup.https.html --ios-simulator

Diff:

--- /Volumes/Data/Builds/GoldenGate/ios/test-280504/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/resource-popup.https-expected.txt
+++ /Volumes/Data/Builds/GoldenGate/ios/test-280504/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/resource-popup.https-actual.txt
@@ -3,16 +3,18 @@
 The HTTP `Refresh` header is used to ensure the child eventually closes itself (since proper observance of COOP will prevent the parent from closing the child in some cases).
 
 
-PASS /common/dummy.xml - parent COOP: ""; child COOP: "same-origin"
-PASS /common/dummy.xml - parent COOP: "same-origin"; child COOP: ""
-PASS /common/dummy.xml - parent COOP: "same-origin"; child COOP: "same-origin"
-PASS /images/red.png - parent COOP: ""; child COOP: "same-origin"
-PASS /images/red.png - parent COOP: "same-origin"; child COOP: ""
-PASS /images/red.png - parent COOP: "same-origin"; child COOP: "same-origin"
-PASS /common/text-plain.txt - parent COOP: ""; child COOP: "same-origin"
-PASS /common/text-plain.txt - parent COOP: "same-origin"; child COOP: ""
-PASS /common/text-plain.txt - parent COOP: "same-origin"; child COOP: "same-origin"
-PASS /media/2x2-green.mp4 - parent COOP: ""; child COOP: "same-origin"
-PASS /media/2x2-green.mp4 - parent COOP: "same-origin"; child COOP: ""
-PASS /media/2x2-green.mp4 - parent COOP: "same-origin"; child COOP: "same-origin"
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT /common/dummy.xml - parent COOP: ""; child COOP: "same-origin" Test timed out
+TIMEOUT /common/dummy.xml - parent COOP: "same-origin"; child COOP: "" Test timed out
+TIMEOUT /common/dummy.xml - parent COOP: "same-origin"; child COOP: "same-origin" Test timed out
+TIMEOUT /images/red.png - parent COOP: ""; child COOP: "same-origin" Test timed out
+TIMEOUT /images/red.png - parent COOP: "same-origin"; child COOP: "" Test timed out
+TIMEOUT /images/red.png - parent COOP: "same-origin"; child COOP: "same-origin" Test timed out
+TIMEOUT /common/text-plain.txt - parent COOP: ""; child COOP: "same-origin" Test timed out
+TIMEOUT /common/text-plain.txt - parent COOP: "same-origin"; child COOP: "" Test timed out
+TIMEOUT /common/text-plain.txt - parent COOP: "same-origin"; child COOP: "same-origin" Test timed out
+TIMEOUT /media/2x2-green.mp4 - parent COOP: ""; child COOP: "same-origin" Test timed out
+TIMEOUT /media/2x2-green.mp4 - parent COOP: "same-origin"; child COOP: "" Test timed out
+TIMEOUT /media/2x2-green.mp4 - parent COOP: "same-origin"; child COOP: "same-origin" Test timed out
+

The test failed at TOT, r280504, and r280110 with the same diff.

[Chris Dumez]

Flaky crash is addressed via Bug 230410. Flaky timeout is not fixed though.