Bug 230691 – [ BigSur wk2 Release ] imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-unsafe-none.https.html is a flaky failure

• Component: New Bugs
• Status: NEW

imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-unsafe-none.https.html

Is a flaky failure on BigSur wk2 Release.

History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fcross-origin-opener-policy%2Fiframe-popup-unsafe-none-to-unsafe-none.https.html

Diff:

--- /Volumes/Data/worker/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-unsafe-none.https-expected.txt
+++ /Volumes/Data/worker/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-unsafe-none.https-actual.txt
@@ -1,3 +1,5 @@
+
+Harness Error (TIMEOUT), message = null
 
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via window_open a SAME_ORIGIN with COOP: ""
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via anchor a SAME_ORIGIN with COOP: ""
@@ -26,7 +28,7 @@
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via window_open a CROSS_ORIGIN with COOP: ""
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via anchor a CROSS_ORIGIN with COOP: ""
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via form_GET a CROSS_ORIGIN with COOP: ""
-PASS unsafe-none with SAME_ORIGIN iframe opening popup via form_POST a CROSS_ORIGIN with COOP: ""
+TIMEOUT unsafe-none with SAME_ORIGIN iframe opening popup via form_POST a CROSS_ORIGIN with COOP: "" Test timed out
 PASS unsafe-none with SAME_SITE iframe opening popup via window_open a CROSS_ORIGIN with COOP: ""
 PASS unsafe-none with SAME_SITE iframe opening popup via anchor a CROSS_ORIGIN with COOP: ""
 PASS unsafe-none with SAME_SITE iframe opening popup via form_GET a CROSS_ORIGIN with COOP: ""

[Radar WebKit Bug Importer]

<rdar://problem/83448682>

Marked test expectations: https://trac.webkit.org/changeset/282962/webkit

I was able to reproduce the failure at TOT using run-webkit-tests --force imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-unsafe-none.https.html --iterations 50 --exit-after-n-failures 1 -f.

It generated a slightly different diff from the one reported above.

--- /Volumes/Data/Builds/GoldenGate/buildToTest/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-unsafe-none.https-expected.txt
+++ /Volumes/Data/Builds/GoldenGate/buildToTest/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-unsafe-none.https-actual.txt
@@ -1,3 +1,5 @@
+
+Harness Error (TIMEOUT), message = null
 
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via window_open a SAME_ORIGIN with COOP: ""
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via anchor a SAME_ORIGIN with COOP: ""
@@ -16,7 +18,7 @@
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via form_GET a SAME_SITE with COOP: ""
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via form_POST a SAME_SITE with COOP: ""
 PASS unsafe-none with SAME_SITE iframe opening popup via window_open a SAME_SITE with COOP: ""
-PASS unsafe-none with SAME_SITE iframe opening popup via anchor a SAME_SITE with COOP: ""
+TIMEOUT unsafe-none with SAME_SITE iframe opening popup via anchor a SAME_SITE with COOP: "" Test timed out
 PASS unsafe-none with SAME_SITE iframe opening popup via form_GET a SAME_SITE with COOP: ""
 PASS unsafe-none with SAME_SITE iframe opening popup via form_POST a SAME_SITE with COOP: ""
 PASS unsafe-none with CROSS_ORIGIN iframe opening popup via window_open a SAME_SITE with COOP: ""

[Chris Dumez]

Yes, I investigated it and we sometimes fail the loads with SSL errors. It doesn't seem related to COOP but rather to the test using HTTPs and doing many loads.

Thank you for your comment Chris. 

I don't think finding a regression point is necessary at this point (?), but I reproduced very similar diffs with revisions as old as r282025. With r282018, the diff is very different (See attached file). I didn't test between r282025 and r282018.

Created attachment 439410 [details]
Diff from r282018

[Chris Dumez]

(In reply to ayumi_kojima from comment #5)
> Thank you for your comment Chris. 
> 
> I don't think finding a regression point is necessary at this point (?), but
> I reproduced very similar diffs with revisions as old as r282025. With
> r282018, the diff is very different (See attached file). I didn't test
> between r282025 and r282018.

No, it isn't necessary.

[Per Arne Vollan]

Pull request: https://github.com/WebKit/WebKit/pull/52529