Bug 228642 - imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-to-unsafe-none.https.html is a flaky failure
Summary: imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: DOM (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Chris Dumez
URL:
Keywords: InRadar
: 228644 229509 (view as bug list)
Depends on:
Blocks:
 
Reported: 2021-07-30 07:46 PDT by Eric Hutchison
Modified: 2021-09-20 08:04 PDT (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Eric Hutchison 2021-07-30 07:46:33 PDT
imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-to-unsafe-none.https.html

is a flaky failure on Mac.

History: https://results.webkit.org/?platform=mac&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fcross-origin-opener-policy%2Fiframe-popup-same-origin-to-unsafe-none.https.html

Results: https://build.webkit.org/results/Apple-BigSur-Release-AppleSilicon-WK2-Tests/r280377%20(2883)/results.html

Diff:
--- /Volumes/Data/worker/bigsur-release-applesilicon-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-to-unsafe-none.https-expected.txt
+++ /Volumes/Data/worker/bigsur-release-applesilicon-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-to-unsafe-none.https-actual.txt
@@ -1,3 +1,5 @@
+
+Harness Error (TIMEOUT), message = null
 
 FAIL same-origin with SAME_ORIGIN iframe opening popup via window_open a SAME_ORIGIN with COOP: "" assert_equals: opener expected false but got true
 FAIL same-origin with SAME_ORIGIN iframe opening popup via anchor a SAME_ORIGIN with COOP: "" assert_equals: opener expected false but got true
@@ -23,7 +25,7 @@
 FAIL same-origin with CROSS_ORIGIN iframe opening popup via anchor a SAME_SITE with COOP: "" assert_equals: opener expected false but got true
 FAIL same-origin with CROSS_ORIGIN iframe opening popup via form_GET a SAME_SITE with COOP: "" assert_equals: opener expected false but got true
 FAIL same-origin with CROSS_ORIGIN iframe opening popup via form_POST a SAME_SITE with COOP: "" assert_equals: opener expected false but got true
-FAIL same-origin with SAME_ORIGIN iframe opening popup via window_open a CROSS_ORIGIN with COOP: "" assert_equals: opener expected false but got true
+TIMEOUT same-origin with SAME_ORIGIN iframe opening popup via window_open a CROSS_ORIGIN with COOP: "" Test timed out
 FAIL same-origin with SAME_ORIGIN iframe opening popup via anchor a CROSS_ORIGIN with COOP: "" assert_equals: opener expected false but got true
 FAIL same-origin with SAME_ORIGIN iframe opening popup via form_GET a CROSS_ORIGIN with COOP: "" assert_equals: opener expected false but got true
 FAIL same-origin with SAME_ORIGIN iframe opening popup via form_POST a CROSS_ORIGIN with COOP: "" assert_equals: opener expected false but got true

Reproduced using run-webkit-tests -f --iterations 500 --exit-after-n-failures 1 --clobber-old-results imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-to-unsafe-none.https.html on r280377.
Comment 1 Radar WebKit Bug Importer 2021-07-30 07:47:15 PDT
<rdar://problem/81325260>
Comment 2 Eric Hutchison 2021-07-30 07:53:16 PDT
Updated test expectations at https://trac.webkit.org/changeset/280473/webkit
Comment 3 Eric Hutchison 2021-09-03 08:59:08 PDT
Issue affects imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-allow-popups-to-same-origin.https.html, imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-same-origin.https.html, and imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-allow-popups-to-same-origin-allow-popups.https.html.

History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fcross-origin-opener-policy%2Fiframe-popup-same-origin-allow-popups-to-same-origin.https.html

Reproduces on r281875.
Comment 4 Eric Hutchison 2021-09-03 08:59:58 PDT
*** Bug 228644 has been marked as a duplicate of this bug. ***
Comment 5 Eric Hutchison 2021-09-03 09:00:20 PDT
*** Bug 229509 has been marked as a duplicate of this bug. ***
Comment 6 Eric Hutchison 2021-09-03 09:11:11 PDT
Updated test expectations for imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-allow-popups-to-same-origin.https.html at http://trac.webkit.org/changeset/281997/webkit.
Comment 7 Eric Hutchison 2021-09-03 09:29:21 PDT
imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-unsafe-none.https.html is also affected.

History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fcross-origin-opener-policy%2Fiframe-popup-unsafe-none-to-unsafe-none.https.html

Update test expectations at http://trac.webkit.org/changeset/282001/webkit.
Comment 8 Chris Dumez 2021-09-03 09:31:29 PDT
Updated diff:

--- /Volumes/Data/worker/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-allow-popups-to-same-origin.https-expected.txt
+++ /Volumes/Data/worker/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-allow-popups-to-same-origin.https-actual.txt
@@ -1,8 +1,10 @@
+
+Harness Error (TIMEOUT), message = null
 
 PASS same-origin-allow-popups with SAME_ORIGIN iframe opening popup via window_open a SAME_ORIGIN with COOP: "same-origin"
 PASS same-origin-allow-popups with SAME_ORIGIN iframe opening popup via anchor a SAME_ORIGIN with COOP: "same-origin"
 PASS same-origin-allow-popups with SAME_ORIGIN iframe opening popup via form_GET a SAME_ORIGIN with COOP: "same-origin"
-PASS same-origin-allow-popups with SAME_ORIGIN iframe opening popup via form_POST a SAME_ORIGIN with COOP: "same-origin"
+TIMEOUT same-origin-allow-popups with SAME_ORIGIN iframe opening popup via form_POST a SAME_ORIGIN with COOP: "same-origin" Test timed out
 PASS same-origin-allow-popups with SAME_SITE iframe opening popup via window_open a SAME_ORIGIN with COOP: "same-origin"
 PASS same-origin-allow-popups with SAME_SITE iframe opening popup via anchor a SAME_ORIGIN with COOP: "same-origin"
 PASS same-origin-allow-popups with SAME_SITE iframe opening popup via form_GET a SAME_ORIGIN with COOP: "same-origin"
Comment 9 Eric Hutchison 2021-09-03 09:40:10 PDT
Also, imported/w3c/web-platform-tests/html/cross-origin-opener-policy/javascript-url.https.html appears to be related with similar diff.

History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fcross-origin-opener-policy%2Fjavascript-url.https.html

Diff: 
--- /Volumes/Data/worker/bigsur-release-applesilicon-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/javascript-url.https-expected.txt
+++ /Volumes/Data/worker/bigsur-release-applesilicon-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/javascript-url.https-actual.txt
@@ -21,6 +21,8 @@
     |  validate    | <--status---+---------------------'
     '--------------'
 
+Harness Error (TIMEOUT), message = null
+
 PASS navigation: SAME_ORIGIN; parentCOOP: unsafe-none; childCOOP: unsafe-none
 PASS navigation: SAME_SITE; parentCOOP: unsafe-none; childCOOP: unsafe-none
 PASS navigation: SAME_ORIGIN; parentCOOP: unsafe-none; childCOOP: same-origin-allow-popups
@@ -36,7 +38,7 @@
 PASS navigation: SAME_ORIGIN; parentCOOP: same-origin; childCOOP: unsafe-none
 PASS navigation: SAME_SITE; parentCOOP: same-origin; childCOOP: unsafe-none
 PASS navigation: SAME_ORIGIN; parentCOOP: same-origin; childCOOP: same-origin-allow-popups
-PASS navigation: SAME_SITE; parentCOOP: same-origin; childCOOP: same-origin-allow-popups
+TIMEOUT navigation: SAME_SITE; parentCOOP: same-origin; childCOOP: same-origin-allow-popups Test timed out
 PASS navigation: SAME_ORIGIN; parentCOOP: same-origin; childCOOP: same-origin
 PASS navigation: SAME_SITE; parentCOOP: same-origin; childCOOP: same-origin
 

Updated test expectations at http://trac.webkit.org/changeset/282002/webkit.
Comment 10 Chris Dumez 2021-09-03 09:41:17 PDT
I was able to reproduce with:
run-webkit-tests --debug --force --repeat-each=100 imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-allow-popups-to-same-origin.https.html

In my case, the failure was:
-PASS same-origin-allow-popups with SAME_ORIGIN iframe opening popup via form_GET a SAME_SITE with COOP: "same-origin"
+TIMEOUT same-origin-allow-popups with SAME_ORIGIN iframe opening popup via form_GET a SAME_SITE with COOP: "same-origin" Test timed out

So form_GET instead of form_POST.
Comment 11 Chris Dumez 2021-09-03 13:10:24 PDT
(In reply to Chris Dumez from comment #10)
> I was able to reproduce with:
> run-webkit-tests --debug --force --repeat-each=100
> imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-
> same-origin-allow-popups-to-same-origin.https.html
> 
> In my case, the failure was:
> -PASS same-origin-allow-popups with SAME_ORIGIN iframe opening popup via
> form_GET a SAME_SITE with COOP: "same-origin"
> +TIMEOUT same-origin-allow-popups with SAME_ORIGIN iframe opening popup via
> form_GET a SAME_SITE with COOP: "same-origin" Test timed out
> 
> So form_GET instead of form_POST.

I just updated my checkout and am no longer able to reproduce anymore.
Comment 12 Chris Dumez 2021-09-07 08:10:47 PDT
No failure since 9/3.
Comment 13 Chris Dumez 2021-09-20 08:04:58 PDT
Unskipped in <https://commits.webkit.org/r282748>.