Bug 232028 - Regression (r284510?): [ iOS macOS wk2 ] imported/w3c/web-platform-tests/html/cross-origin-opener-policy/reporting/navigation-reporting/report-only-four-reports.https.html is a flaky failure
Summary: Regression (r284510?): [ iOS macOS wk2 ] imported/w3c/web-platform-tests/html...
Status: RESOLVED DUPLICATE of bug 234899
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-10-20 11:28 PDT by ayumi_kojima
Modified: 2022-01-06 09:29 PST (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description ayumi_kojima 2021-10-20 11:28:01 PDT
imported/w3c/web-platform-tests/html/cross-origin-opener-policy/reporting/navigation-reporting/report-only-four-reports.https.html

Is a flaky failure on iOS 15 and macOS wk2.

History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fcross-origin-opener-policy%2Freporting%2Fnavigation-reporting%2Freport-only-four-reports.https.html

Result page: https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r284525%20(239)/results.html

Diff:

--- /Volumes/Data/worker/ios-simulator-15-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/reporting/navigation-reporting/report-only-four-reports.https-expected.txt
+++ /Volumes/Data/worker/ios-simulator-15-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/reporting/navigation-reporting/report-only-four-reports.https-actual.txt
@@ -1,4 +1,4 @@
 
-PASS coop reporting test A test with both COOP and COOP report only setup to CROSS_ORIGIN with same-origin-allow-popups; report-to="coop-popup-report-endpoint", require-corp, same-origin; report-to="coop-popup-report-only-endpoint", require-corp
-PASS verify remaining reports
+FAIL coop reporting test A test with both COOP and COOP report only setup to CROSS_ORIGIN with same-origin-allow-popups; report-to="coop-popup-report-endpoint", require-corp, same-origin; report-to="coop-popup-report-only-endpoint", require-corp promise_test: Unhandled rejection with value: "No report matched the expected report for endpoint: coop-popup-report-endpoint, expected report: {\"body\":{\"disposition\":\"enforce\",\"effectivePolicy\":\"same-origin-allow-popups\",\"previousResponseURL\":\"\",\"referrer\":\"https://localhost:9443/\",\"type\":\"navigation-to-response\"},\"url\":\"/uuid=(uuid)$/\",\"type\":\"coop\"}, within available reports: [{\"type\":\"coop\",\"url\":\"https://127.0.0.1:9443/common/dispatcher/executor.html?pipe=|header(report-to,%7B%22group%22%3A%22coop-report-endpoint%22%5C%2C%22max_age%22%3A3600%5C%2C%22endpoints%22%3A%5B%7B%22url%22%3A%22https%3A%2F%2F127.0.0.1%3A9443%2Freporting%2Fresources%2Freport.py%3FreportID%3D(uuid)%22%7D%5D%7D%5C%2C%7B%22group%22%3A%22coop-report-only-endpoint%22%5C%2C%22max_age%22%3A3600%5C%2C%22endpoints%22%3A%5B%7B%22url%22%3A%22https%3A%2F%2F127.0.0.1%3A9443%2Freporting%2Fresources%2Freport.py%3FreportID%3D(uuid)%22%7D%5D%7D%5C%2C%7B%22group%22%3A%22coop-popup-report-endpoint%22%5C%2C%22max_age%22%3A3600%5C%2C%22endpoints%22%3A%5B%7B%22url%22%3A%22https%3A%2F%2F127.0.0.1%3A9443%2Freporting%2Fresources%2Freport.py%3FreportID%3D(uuid)%22%7D%5D%7D%5C%2C%7B%22group%22%3A%22coop-popup-report-only-endpoint%22%5C%2C%22max_age%22%3A3600%5C%2C%22endpoints%22%3A%5B%7B%22url%22%3A%22https%3A%2F%2F127.0.0.1%3A9443%2Freporting%2Fresources%2Freport.py%3FreportID%3D(uuid)%22%7D%5D%7D%5C%2C%7B%22group%22%3A%22coop-redirect-report-endpoint%22%5C%2C%22max_age%22%3A3600%5C%2C%22endpoints%22%3A%5B%7B%22url%22%3A%22https%3A%2F%2F127.0.0.1%3A9443%2Freporting%2Fresources%2Freport.py%3FreportID%3D(uuid)%22%7D%5D%7D%5C%2C%7B%22group%22%3A%22coop-redirect-report-only-endpoint%22%5C%2C%22max_age%22%3A3600%5C%2C%22endpoints%22%3A%5B%7B%22url%22%3A%22https%3A%2F%2F127.0.0.1%3A9443%2Freporting%2Fresources%2Freport.py%3FreportID%3D(uuid)%22%7D%5D%7D)|header(Cross-Origin-Opener-Policy,same-origin-allow-popups%3B%20report-to%3D%22coop-popup-report-endpoint%22)|header(Cross-Origin-Embedder-Policy,require-corp)|header(Cross-Origin-Opener-Policy-Report-Only,same-origin%3B%20report-to%3D%22coop-popup-report-only-endpoint%22)|header(Cross-Origin-Embedder-Policy-Report-Only,require-corp)&uuid=(uuid)\",\"user_agent\":\"Mozilla/5.0 (iPhone; CPU iPhone OS 15_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148\",\"age\":0,\"body\":{\"disposition\":\"enforce\",\"effectivePolicy\":\"same-origin-allow-popups\",\"previousResponseURL\":\"\",\"type\":\"navigation-to-response\",\"referrer\":\"https://localhost:9443/\"},\"metadata\":{\"content_type\":\"application/reports+json\"}}]"
+FAIL verify remaining reports assert_equals: coop-popup-report-endpoint should be empty expected 0 but got 1
Comment 1 Radar WebKit Bug Importer 2021-10-20 11:43:20 PDT
<rdar://problem/84472339>
Comment 2 ayumi_kojima 2021-10-20 11:43:47 PDT
Marked test expectations https://trac.webkit.org/changeset/284555/webkit
Comment 3 ayumi_kojima 2021-10-20 11:59:08 PDT
I was able to reproduce the failure locally using run-webkit-tests --ios-simulator --iterations 50 --exit-after-n-failures 1 imported/w3c/web-platform-tests/html/cross-origin-opener-policy/reporting/navigation-reporting/report-only-four-reports.https.html -f
Comment 4 ayumi_kojima 2021-10-20 12:01:16 PDT
The test failed at r284510. No revision available to test for r284509, but I think the failure started at https://trac.webkit.org/changeset/284510/webkit.
Comment 5 Chris Dumez 2021-10-21 10:20:35 PDT
(In reply to ayumi_kojima from comment #4)
> The test failed at r284510. No revision available to test for r284509, but I
> think the failure started at https://trac.webkit.org/changeset/284510/webkit.

r284510 is when I resync'd the test from upstream. I'll see what changes got merged for this particular test.
Comment 6 Chris Dumez 2021-10-21 11:20:21 PDT
Looks like I was able to reproduce as well:

--- /Volumes/Data/WebKit/OpenSource/WebKitBuild/Release-iphonesimulator/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/reporting/navigation-reporting/report-only-four-reports.https-expected.txt
+++ /Volumes/Data/WebKit/OpenSource/WebKitBuild/Release-iphonesimulator/layout-test-results/imported/w3c/web-platform-tests/html/cross-origin-opener-policy/reporting/navigation-reporting/report-only-four-reports.https-actual.txt
@@ -1,4 +1,4 @@
 
-PASS coop reporting test A test with both COOP and COOP report only setup to CROSS_ORIGIN with same-origin-allow-popups; report-to="coop-popup-report-endpoint", require-corp, same-origin; report-to="coop-popup-report-only-endpoint", require-corp
+FAIL coop reporting test A test with both COOP and COOP report only setup to CROSS_ORIGIN with same-origin-allow-popups; report-to="coop-popup-report-endpoint", require-corp, same-origin; report-to="coop-popup-report-only-endpoint", require-corp promise_test: Unhandled rejection with value: "No report matched the expected report for endpoint: coop-report-endpoint, expected report: {\"body\":{\"disposition\":\"enforce\",\"effectivePolicy\":\"same-origin-allow-popups\",\"nextResponseURL\":\"/uuid=(uuid)$/\",\"type\":\"navigation-from-response\"},\"url\":\"https://localhost:9443/html/cross-origin-opener-policy/reporting/navigation-reporting/report-only-four-reports.https.html\",\"type\":\"coop\"}, within available reports: []"
 PASS verify remaining reports
Comment 7 Chris Dumez 2022-01-06 09:29:45 PST

*** This bug has been marked as a duplicate of bug 234899 ***