Bug 266047 – REGRESSION(266644@main):[ macOS WK1 ] imported/w3c/web-platform-tests/content-security-policy/reporting/report-original-url-on-mixed-content-frame.https.sub.html is a constant text failure

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Robert Jenner]

imported/w3c/web-platform-tests/content-security-policy/reporting/report-original-url-on-mixed-content-frame.https.sub.html

is a constant text failure on macOS WK 1 only.

HISTORY:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fcontent-security-policy%2Freporting%2Freport-original-url-on-mixed-content-frame.https.sub.html

DIFF TEXT:
+CONSOLE MESSAGE: Blocked mixed content http://localhost:8800/content-security-policy/support/fail.html?t=1 because 'block-all-mixed-content' appears in the Content Security Policy.
 
-FAIL Violation report status OK. assert_true: blocked-uri value of  "http://localhost:8800" did not match https://localhost:9443/common/redirect.py?location=http%3A%2F%2Flocalhost%3A8800%2Fcontent-security-policy%2Fsupport%2Ffail.html%3Ft%3D1. expected true got false
+FAIL Violation report status OK. undefined is not an object (evaluating 'data[0]["body"]')

DIFF URL:
https://build.webkit.org/results/Apple-Ventura-Release-AppleSilicon-WK1-Tests/271700@main%20(6588)/imported/w3c/web-platform-tests/content-security-policy/reporting/report-original-url-on-mixed-content-frame.https.sub-pretty-diff.html

[Radar WebKit Bug Importer]

<rdar://problem/119349300>

[Robert Jenner]

This is easily reproducible at Sonoma Release WK1 ToT running the test as follows:

run-webkit-tests imported/w3c/web-platform-tests/content-security-policy/reporting/report-original-url-on-mixed-content-frame.https.sub.html -1 --force


There's already an expectation marked for this test because it was slowing down EWS. It appears that this test has been failing on macOS WK1 only for at least 50,000 commits if not more.

[Robert Jenner]

My mistake, I thought this had been failing for longer than it has been. As it turns out, I was able to discover a regression point. https://commits.webkit.org/266644@main is what broke this test

I verified it by checking out my head back to 266643@main, and running the test on a ToT spade..

[Robert Jenner]

(In reply to Robert Jenner from comment #3)
> My mistake, I thought this had been failing for longer than it has been. As
> it turns out, I was able to discover a regression point.
> https://commits.webkit.org/266644@main is what broke this test
> 
> I verified it by checking out my head back to 266643@main, and running the
> test on a ToT build..

Sorry, unfinished thought here. Running the repo steps above with your HEAD checked out to 266643@main, but running testing on the ToT build produces a pass. Checking out your HEAD to 266644@main, and running the tests again on the same ToT build produces the failure. So https://commits.webkit.org/266644@main is the culprit.

[Brent Fulgham]

The original result for LayoutTests/imported/w3c/web-platform-tests/content-security-policy/reporting/report-original-url-on-mixed-content-frame.https.sub-expected.txt was the result produced by WebKitLegacy. Either I had previously checked in a bad result for modern WebKit and didn’t notice it, or Ryan fixed a bug that was preventing the test from passing on all platforms.

Either way, Ryan's work in 266644@main progressed this test case in modern WebKit, and he checked that expectation in.

Unfortunately, WebKitLegacy relies on a PingHandle implementation that cannot authenticate against https (which is a missing feature in the entire deprecated WebKitLegacy platform for the old PingHandle implementation). We do to intend to invest time working on implementing such a feature in WebKitLegacy, so expect this test to fail.

In the end, this was indeed just a Test Development bug. I need to check the correct expectation in for WebKitLegacy at which point the test system will be in the correct state.

I'll land a WebKitLegacy test expectation so we can document the expected behavior. Note that the new logging shows that the load was blocked properly in WebKitLegacy. However, the report message never gets to the https endpoint because it needs the non-existant `canAuthenticateAgainstProtectionSpace` logic to work properly. (People: Don't use WebKitLegacy for things!)

[Brent Fulgham]

PR: https://github.com/WebKit/WebKit/pull/28315

[EWS]

Committed 278543@main (c741b1a50100): <https://commits.webkit.org/278543@main>

Reviewed commits have been landed. Closing PR #28315 and removing active labels.