Bug 230428

Summary: [ MacOS ] http/tests/security/contentSecurityPolicy/frame-src-cross-origin-load.html is a flakey failure
Product: WebKit Reporter: ayumi_kojima
Component: New BugsAssignee: Kate Cheney <katherine_cheney>
Status: RESOLVED FIXED    
Severity: Normal CC: bfulgham, ews-watchlist, katherine_cheney, mkwst, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=222748
Attachments:
Description Flags
Patch none

Description ayumi_kojima 2021-09-17 13:57:44 PDT 
http/tests/security/contentSecurityPolicy/frame-src-cross-origin-load.html

Is a flaky failure on macOS. It is a flaky failure/timeout on iOS.

The test has been flaky on macOS Debug (bug 222748), but now it is flaky on both Release and Debug.

History: https://results.webkit.org/?suite=layout-tests&test=http%2Ftests%2Fsecurity%2FcontentSecurityPolicy%2Fframe-src-cross-origin-load.html

Result page: https://build.webkit.org/results/Apple-BigSur-Release-WK1-Tests/r282556%20(5339)/results.html

Diff:

--- /Volumes/Data/worker/bigsur-release-tests-wk1/build/layout-test-results/http/tests/security/contentSecurityPolicy/frame-src-cross-origin-load-expected.txt
+++ /Volumes/Data/worker/bigsur-release-tests-wk1/build/layout-test-results/http/tests/security/contentSecurityPolicy/frame-src-cross-origin-load-actual.txt
@@ -1,6 +1,6 @@
 CONSOLE MESSAGE: Refused to load https://localhost:8443/security/contentSecurityPolicy/resources/alert-fail.html because it does not appear in the frame-src directive of the Content Security Policy.
+CONSOLE MESSAGE: PASS
 ALERT: PASS
-CONSOLE MESSAGE: PASS
 IFrames blocked by CSP should generate a 'load' event, regardless of blocked state. This means they appear to be normal cross-origin loads, thereby not leaking URL information directly to JS.
 
 On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
Comment 1 Radar WebKit Bug Importer 2021-09-17 13:58:14 PDT 
<rdar://problem/83254559>
Comment 2 ayumi_kojima 2021-09-17 14:13:58 PDT 
For iOS, the test is mostly flaky on Release, but I see it timed out on Debug one time. Market test expectations: https://trac.webkit.org/changeset/282692/webkit
Comment 3 ayumi_kojima 2021-10-14 14:58:02 PDT 
I don't see timeouts any more on the recent results. Also, the flaky failure is only seen on macOS (mainly wk1).

I was not able to reproduce the failure locally on BigSure using run-webkit-tests -debug -1 --force --iterations 1000 --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 http/tests/security/contentSecurityPolicy/frame-src-cross-origin-load.html -f
Comment 4 ayumi_kojima 2021-10-14 15:05:15 PDT 
Removed test expectation for iOS https://trac.webkit.org/changeset/284200/webkit
Comment 5 Kate Cheney 2022-03-14 17:38:27 PDT 
Created attachment 454641 [details]
Patch
Comment 6 Brent Fulgham 2022-03-16 10:55:22 PDT 
Comment on attachment 454641 [details]
Patch

Nice solution! r=me
Comment 7 EWS 2022-03-16 11:43:55 PDT 
Committed r291359 (248491@main): <https://commits.webkit.org/248491@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 454641 [details].