Bug 230011

Summary:	[ iOS EWS ] imported/w3c/web-platform-tests/html/cross-origin-opener-policy/resource-popup.https.html is flaky
Product:	WebKit	Reporter:	ayumi_kojima
Component:	DOM	Assignee:	Nobody <webkit-unassigned>
Status:	NEW ---
Severity:	Normal	CC:	ap, cdumez, jer.noble, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	iPhone / iPad
OS:	Unspecified
See Also:	https://bugs.webkit.org/show_bug.cgi?id=228642 https://bugs.webkit.org/show_bug.cgi?id=230410

Description ayumi_kojima 2021-09-07 10:53:01 PDT

imported/w3c/web-platform-tests/html/cross-origin-opener-policy/resource-popup.https.html

Is flaky crashing on iOS-14-Simulator-WK2-Tests-EWS

The test started showing up as a flaky test at https://ews-build.webkit.org/#/builders/51/builds/20556

The test seems to have been a flaky failure/ crash in the open source directory: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fcross-origin-opener-policy%2Fresource-popup.https.html

Crash log:

Failed to find media engine.
com.apple.WebKit.WebContent.Development terminated (pid 22070) because the process crashed

Comment 1 Radar WebKit Bug Importer 2021-09-07 10:53:32 PDT

<rdar://problem/82829778>

Comment 2 ayumi_kojima 2021-09-07 10:56:17 PDT

Updated test expectations to speed up EWS https://trac.webkit.org/changeset/282095/webkit

Comment 3 Alexey Proskuryakov 2021-09-12 13:34:01 PDT

This is a test for COOP.

Comment 4 ayumi_kojima 2021-09-13 14:24:36 PDT

The test is flaky failing on BigSur wk2 Release as well.

Diff:

--- /Volumes/Data/worker/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/resource-popup.https-expected.txt
+++ /Volumes/Data/worker/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/resource-popup.https-actual.txt
@@ -2,6 +2,8 @@
 
 The HTTP `Refresh` header is used to ensure the child eventually closes itself (since proper observance of COOP will prevent the parent from closing the child in some cases).
 
+
+Harness Error (TIMEOUT), message = null
 
 PASS /common/dummy.xml - parent COOP: ""; child COOP: "same-origin"
 PASS /common/dummy.xml - parent COOP: "same-origin"; child COOP: ""
@@ -9,7 +11,7 @@
 PASS /images/red.png - parent COOP: ""; child COOP: "same-origin"
 PASS /images/red.png - parent COOP: "same-origin"; child COOP: ""
 PASS /images/red.png - parent COOP: "same-origin"; child COOP: "same-origin"
-PASS /common/text-plain.txt - parent COOP: ""; child COOP: "same-origin"
+TIMEOUT /common/text-plain.txt - parent COOP: ""; child COOP: "same-origin" Test timed out
 PASS /common/text-plain.txt - parent COOP: "same-origin"; child COOP: ""
 PASS /common/text-plain.txt - parent COOP: "same-origin"; child COOP: "same-origin"
 PASS /media/2x2-green.mp4 - parent COOP: ""; child COOP: "same-origin"

Comment 5 ayumi_kojima 2021-09-13 14:25:21 PDT

From the test name and diff, this might be related to Bug 228642.

Comment 6 ayumi_kojima 2021-09-13 14:55:02 PDT

I was not able to reproduce the failure locally on iOS 14 simulator using run-webkit-tests --force --iterations 100 --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 imported/w3c/web-platform-tests/html/cross-origin-opener-policy/resource-popup.https.html --ios-simulator

Diff:

--- /Volumes/Data/Builds/GoldenGate/ios/test-280504/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/resource-popup.https-expected.txt
+++ /Volumes/Data/Builds/GoldenGate/ios/test-280504/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/resource-popup.https-actual.txt
@@ -3,16 +3,18 @@
 The HTTP `Refresh` header is used to ensure the child eventually closes itself (since proper observance of COOP will prevent the parent from closing the child in some cases).
 
 
-PASS /common/dummy.xml - parent COOP: ""; child COOP: "same-origin"
-PASS /common/dummy.xml - parent COOP: "same-origin"; child COOP: ""
-PASS /common/dummy.xml - parent COOP: "same-origin"; child COOP: "same-origin"
-PASS /images/red.png - parent COOP: ""; child COOP: "same-origin"
-PASS /images/red.png - parent COOP: "same-origin"; child COOP: ""
-PASS /images/red.png - parent COOP: "same-origin"; child COOP: "same-origin"
-PASS /common/text-plain.txt - parent COOP: ""; child COOP: "same-origin"
-PASS /common/text-plain.txt - parent COOP: "same-origin"; child COOP: ""
-PASS /common/text-plain.txt - parent COOP: "same-origin"; child COOP: "same-origin"
-PASS /media/2x2-green.mp4 - parent COOP: ""; child COOP: "same-origin"
-PASS /media/2x2-green.mp4 - parent COOP: "same-origin"; child COOP: ""
-PASS /media/2x2-green.mp4 - parent COOP: "same-origin"; child COOP: "same-origin"
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT /common/dummy.xml - parent COOP: ""; child COOP: "same-origin" Test timed out
+TIMEOUT /common/dummy.xml - parent COOP: "same-origin"; child COOP: "" Test timed out
+TIMEOUT /common/dummy.xml - parent COOP: "same-origin"; child COOP: "same-origin" Test timed out
+TIMEOUT /images/red.png - parent COOP: ""; child COOP: "same-origin" Test timed out
+TIMEOUT /images/red.png - parent COOP: "same-origin"; child COOP: "" Test timed out
+TIMEOUT /images/red.png - parent COOP: "same-origin"; child COOP: "same-origin" Test timed out
+TIMEOUT /common/text-plain.txt - parent COOP: ""; child COOP: "same-origin" Test timed out
+TIMEOUT /common/text-plain.txt - parent COOP: "same-origin"; child COOP: "" Test timed out
+TIMEOUT /common/text-plain.txt - parent COOP: "same-origin"; child COOP: "same-origin" Test timed out
+TIMEOUT /media/2x2-green.mp4 - parent COOP: ""; child COOP: "same-origin" Test timed out
+TIMEOUT /media/2x2-green.mp4 - parent COOP: "same-origin"; child COOP: "" Test timed out
+TIMEOUT /media/2x2-green.mp4 - parent COOP: "same-origin"; child COOP: "same-origin" Test timed out
+

The test failed at TOT, r280504, and r280110 with the same diff.

Comment 7 Chris Dumez 2021-09-17 09:35:21 PDT

Flaky crash is addressed via Bug 230410. Flaky timeout is not fixed though.