Bug 228642 – imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-to-unsafe-none.https.html is a flaky failure

• Status: RESOLVED
• Resolution: FIXED
• Component: DOM

[Eric Hutchison]

imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-to-unsafe-none.https.html

is a flaky failure on Mac.

History: https://results.webkit.org/?platform=mac&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fcross-origin-opener-policy%2Fiframe-popup-same-origin-to-unsafe-none.https.html

Results: https://build.webkit.org/results/Apple-BigSur-Release-AppleSilicon-WK2-Tests/r280377%20(2883)/results.html

Diff:
--- /Volumes/Data/worker/bigsur-release-applesilicon-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-to-unsafe-none.https-expected.txt
+++ /Volumes/Data/worker/bigsur-release-applesilicon-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-to-unsafe-none.https-actual.txt
@@ -1,3 +1,5 @@
+
+Harness Error (TIMEOUT), message = null
 
 FAIL same-origin with SAME_ORIGIN iframe opening popup via window_open a SAME_ORIGIN with COOP: "" assert_equals: opener expected false but got true
 FAIL same-origin with SAME_ORIGIN iframe opening popup via anchor a SAME_ORIGIN with COOP: "" assert_equals: opener expected false but got true
@@ -23,7 +25,7 @@
 FAIL same-origin with CROSS_ORIGIN iframe opening popup via anchor a SAME_SITE with COOP: "" assert_equals: opener expected false but got true
 FAIL same-origin with CROSS_ORIGIN iframe opening popup via form_GET a SAME_SITE with COOP: "" assert_equals: opener expected false but got true
 FAIL same-origin with CROSS_ORIGIN iframe opening popup via form_POST a SAME_SITE with COOP: "" assert_equals: opener expected false but got true
-FAIL same-origin with SAME_ORIGIN iframe opening popup via window_open a CROSS_ORIGIN with COOP: "" assert_equals: opener expected false but got true
+TIMEOUT same-origin with SAME_ORIGIN iframe opening popup via window_open a CROSS_ORIGIN with COOP: "" Test timed out
 FAIL same-origin with SAME_ORIGIN iframe opening popup via anchor a CROSS_ORIGIN with COOP: "" assert_equals: opener expected false but got true
 FAIL same-origin with SAME_ORIGIN iframe opening popup via form_GET a CROSS_ORIGIN with COOP: "" assert_equals: opener expected false but got true
 FAIL same-origin with SAME_ORIGIN iframe opening popup via form_POST a CROSS_ORIGIN with COOP: "" assert_equals: opener expected false but got true

Reproduced using run-webkit-tests -f --iterations 500 --exit-after-n-failures 1 --clobber-old-results imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-to-unsafe-none.https.html on r280377.

[Radar WebKit Bug Importer]

<rdar://problem/81325260>

[Eric Hutchison]

Updated test expectations at https://trac.webkit.org/changeset/280473/webkit

[Eric Hutchison]

Issue affects imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-allow-popups-to-same-origin.https.html, imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-same-origin.https.html, and imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-allow-popups-to-same-origin-allow-popups.https.html.

History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fcross-origin-opener-policy%2Fiframe-popup-same-origin-allow-popups-to-same-origin.https.html

Reproduces on r281875.

[Eric Hutchison]

*** Bug 228644 has been marked as a duplicate of this bug. ***

[Eric Hutchison]

*** Bug 229509 has been marked as a duplicate of this bug. ***

[Eric Hutchison]

Updated test expectations for imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-allow-popups-to-same-origin.https.html at http://trac.webkit.org/changeset/281997/webkit.

[Eric Hutchison]

imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-unsafe-none-to-unsafe-none.https.html is also affected.

History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fcross-origin-opener-policy%2Fiframe-popup-unsafe-none-to-unsafe-none.https.html

Update test expectations at http://trac.webkit.org/changeset/282001/webkit.

[Chris Dumez]

Updated diff:

--- /Volumes/Data/worker/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-allow-popups-to-same-origin.https-expected.txt
+++ /Volumes/Data/worker/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-allow-popups-to-same-origin.https-actual.txt
@@ -1,8 +1,10 @@
+
+Harness Error (TIMEOUT), message = null
 
 PASS same-origin-allow-popups with SAME_ORIGIN iframe opening popup via window_open a SAME_ORIGIN with COOP: "same-origin"
 PASS same-origin-allow-popups with SAME_ORIGIN iframe opening popup via anchor a SAME_ORIGIN with COOP: "same-origin"
 PASS same-origin-allow-popups with SAME_ORIGIN iframe opening popup via form_GET a SAME_ORIGIN with COOP: "same-origin"
-PASS same-origin-allow-popups with SAME_ORIGIN iframe opening popup via form_POST a SAME_ORIGIN with COOP: "same-origin"
+TIMEOUT same-origin-allow-popups with SAME_ORIGIN iframe opening popup via form_POST a SAME_ORIGIN with COOP: "same-origin" Test timed out
 PASS same-origin-allow-popups with SAME_SITE iframe opening popup via window_open a SAME_ORIGIN with COOP: "same-origin"
 PASS same-origin-allow-popups with SAME_SITE iframe opening popup via anchor a SAME_ORIGIN with COOP: "same-origin"
 PASS same-origin-allow-popups with SAME_SITE iframe opening popup via form_GET a SAME_ORIGIN with COOP: "same-origin"

[Eric Hutchison]

Also, imported/w3c/web-platform-tests/html/cross-origin-opener-policy/javascript-url.https.html appears to be related with similar diff.

History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fcross-origin-opener-policy%2Fjavascript-url.https.html

Diff: 
--- /Volumes/Data/worker/bigsur-release-applesilicon-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/javascript-url.https-expected.txt
+++ /Volumes/Data/worker/bigsur-release-applesilicon-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/javascript-url.https-actual.txt
@@ -21,6 +21,8 @@
     |  validate    | <--status---+---------------------'
     '--------------'
 
+Harness Error (TIMEOUT), message = null
+
 PASS navigation: SAME_ORIGIN; parentCOOP: unsafe-none; childCOOP: unsafe-none
 PASS navigation: SAME_SITE; parentCOOP: unsafe-none; childCOOP: unsafe-none
 PASS navigation: SAME_ORIGIN; parentCOOP: unsafe-none; childCOOP: same-origin-allow-popups
@@ -36,7 +38,7 @@
 PASS navigation: SAME_ORIGIN; parentCOOP: same-origin; childCOOP: unsafe-none
 PASS navigation: SAME_SITE; parentCOOP: same-origin; childCOOP: unsafe-none
 PASS navigation: SAME_ORIGIN; parentCOOP: same-origin; childCOOP: same-origin-allow-popups
-PASS navigation: SAME_SITE; parentCOOP: same-origin; childCOOP: same-origin-allow-popups
+TIMEOUT navigation: SAME_SITE; parentCOOP: same-origin; childCOOP: same-origin-allow-popups Test timed out
 PASS navigation: SAME_ORIGIN; parentCOOP: same-origin; childCOOP: same-origin
 PASS navigation: SAME_SITE; parentCOOP: same-origin; childCOOP: same-origin
 

Updated test expectations at http://trac.webkit.org/changeset/282002/webkit.

[Chris Dumez]

I was able to reproduce with:
run-webkit-tests --debug --force --repeat-each=100 imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-allow-popups-to-same-origin.https.html

In my case, the failure was:
-PASS same-origin-allow-popups with SAME_ORIGIN iframe opening popup via form_GET a SAME_SITE with COOP: "same-origin"
+TIMEOUT same-origin-allow-popups with SAME_ORIGIN iframe opening popup via form_GET a SAME_SITE with COOP: "same-origin" Test timed out

So form_GET instead of form_POST.

[Chris Dumez]

(In reply to Chris Dumez from comment #10)
> I was able to reproduce with:
> run-webkit-tests --debug --force --repeat-each=100
> imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-
> same-origin-allow-popups-to-same-origin.https.html
> 
> In my case, the failure was:
> -PASS same-origin-allow-popups with SAME_ORIGIN iframe opening popup via
> form_GET a SAME_SITE with COOP: "same-origin"
> +TIMEOUT same-origin-allow-popups with SAME_ORIGIN iframe opening popup via
> form_GET a SAME_SITE with COOP: "same-origin" Test timed out
> 
> So form_GET instead of form_POST.

I just updated my checkout and am no longer able to reproduce anymore.

[Chris Dumez]

No failure since 9/3.

[Chris Dumez]

Unskipped in <https://commits.webkit.org/r282748>.