Bug 230691 - [ BigSur wk2 Release ] imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-unsafe-none.https.html is a flaky failure
Summary: [ BigSur wk2 Release ] imported/w3c/web-platform-tests/html/cross-origin-open...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-09-23 08:40 PDT by ayumi_kojima
Modified: 2021-09-27 16:20 PDT (History)
3 users (show)

See Also:


Attachments
Diff from r282018 (7.88 KB, text/plain)
2021-09-27 16:18 PDT, ayumi_kojima
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description ayumi_kojima 2021-09-23 08:40:03 PDT
imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-unsafe-none.https.html

Is a flaky failure on BigSur wk2 Release.

History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fcross-origin-opener-policy%2Fiframe-popup-unsafe-none-to-unsafe-none.https.html

Diff:

--- /Volumes/Data/worker/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-unsafe-none.https-expected.txt
+++ /Volumes/Data/worker/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-unsafe-none.https-actual.txt
@@ -1,3 +1,5 @@
+
+Harness Error (TIMEOUT), message = null
 
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via window_open a SAME_ORIGIN with COOP: ""
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via anchor a SAME_ORIGIN with COOP: ""
@@ -26,7 +28,7 @@
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via window_open a CROSS_ORIGIN with COOP: ""
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via anchor a CROSS_ORIGIN with COOP: ""
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via form_GET a CROSS_ORIGIN with COOP: ""
-PASS unsafe-none with SAME_ORIGIN iframe opening popup via form_POST a CROSS_ORIGIN with COOP: ""
+TIMEOUT unsafe-none with SAME_ORIGIN iframe opening popup via form_POST a CROSS_ORIGIN with COOP: "" Test timed out
 PASS unsafe-none with SAME_SITE iframe opening popup via window_open a CROSS_ORIGIN with COOP: ""
 PASS unsafe-none with SAME_SITE iframe opening popup via anchor a CROSS_ORIGIN with COOP: ""
 PASS unsafe-none with SAME_SITE iframe opening popup via form_GET a CROSS_ORIGIN with COOP: ""
Comment 1 Radar WebKit Bug Importer 2021-09-23 08:40:28 PDT
<rdar://problem/83448682>
Comment 2 ayumi_kojima 2021-09-23 08:50:24 PDT
Marked test expectations: https://trac.webkit.org/changeset/282962/webkit
Comment 3 ayumi_kojima 2021-09-27 14:35:52 PDT
I was able to reproduce the failure at TOT using run-webkit-tests --force imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-unsafe-none.https.html --iterations 50 --exit-after-n-failures 1 -f.

It generated a slightly different diff from the one reported above.

--- /Volumes/Data/Builds/GoldenGate/buildToTest/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-unsafe-none.https-expected.txt
+++ /Volumes/Data/Builds/GoldenGate/buildToTest/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-unsafe-none.https-actual.txt
@@ -1,3 +1,5 @@
+
+Harness Error (TIMEOUT), message = null
 
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via window_open a SAME_ORIGIN with COOP: ""
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via anchor a SAME_ORIGIN with COOP: ""
@@ -16,7 +18,7 @@
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via form_GET a SAME_SITE with COOP: ""
 PASS unsafe-none with SAME_ORIGIN iframe opening popup via form_POST a SAME_SITE with COOP: ""
 PASS unsafe-none with SAME_SITE iframe opening popup via window_open a SAME_SITE with COOP: ""
-PASS unsafe-none with SAME_SITE iframe opening popup via anchor a SAME_SITE with COOP: ""
+TIMEOUT unsafe-none with SAME_SITE iframe opening popup via anchor a SAME_SITE with COOP: "" Test timed out
 PASS unsafe-none with SAME_SITE iframe opening popup via form_GET a SAME_SITE with COOP: ""
 PASS unsafe-none with SAME_SITE iframe opening popup via form_POST a SAME_SITE with COOP: ""
 PASS unsafe-none with CROSS_ORIGIN iframe opening popup via window_open a SAME_SITE with COOP: ""
Comment 4 Chris Dumez 2021-09-27 14:40:38 PDT
Yes, I investigated it and we sometimes fail the loads with SSL errors. It doesn't seem related to COOP but rather to the test using HTTPs and doing many loads.
Comment 5 ayumi_kojima 2021-09-27 16:17:29 PDT
Thank you for your comment Chris. 

I don't think finding a regression point is necessary at this point (?), but I reproduced very similar diffs with revisions as old as r282025. With r282018, the diff is very different (See attached file). I didn't test between r282025 and r282018.
Comment 6 ayumi_kojima 2021-09-27 16:18:05 PDT
Created attachment 439410 [details]
Diff from r282018
Comment 7 Chris Dumez 2021-09-27 16:20:32 PDT
(In reply to ayumi_kojima from comment #5)
> Thank you for your comment Chris. 
> 
> I don't think finding a regression point is necessary at this point (?), but
> I reproduced very similar diffs with revisions as old as r282025. With
> r282018, the diff is very different (See attached file). I didn't test
> between r282025 and r282018.

No, it isn't necessary.