Bug 245967 – New Tests(255047@main): [ iOS macOS ] 3X imported/w3c/web-platform-tests/feature-policy/(Layout tests) are constantly failing

• Component: New Bugs
• Status: RESOLVED FIXED

[Hercules Hjalmarsson]

imported/w3c/web-platform-tests/feature-policy/reporting/picture-in-picture-reporting.html
imported/w3c/web-platform-tests/feature-policy/feature-policy-frame-policy-allowed-for-some-override.https.sub.html
imported/w3c/web-platform-tests/feature-policy/experimental-features/vertical-scroll-scrollintoview.tentative.html 

Are three constant text failures on iOS and macOS ToT and since introduced at 255047@main. I'm unsure if some of these are simple rebaselines since they clearly do not meet expected output.

HISTORY:
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Ffeature-policy%2Freporting%2Fpicture-in-picture-reporting.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fscreen-orientation%2Forientation-reading.html&test=imported%2Fw3c%2Fweb-platform-tests%2Ffeature-policy%2Fexperimental-features%2Fvertical-scroll-scrollintoview.tentative.html

DIFFS:

--- /Volumes/Data/worker/Apple-iOS-16-Simulator-Release-arm64-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/feature-policy/experimental-features/vertical-scroll-scrollintoview.tentative-expected.txt
+++ /Volumes/Data/worker/Apple-iOS-16-Simulator-Release-arm64-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/feature-policy/experimental-features/vertical-scroll-scrollintoview.tentative-actual.txt
@@ -5,5 +5,5 @@
 
 
 PASS Calling 'scrollIntoView()' inside a <iframe> will propagate up by default('vertical-scroll' enabled).
-FAIL Calling 'scrollIntoView()' inside a <iframe> with 'vertical-scroll none;'will not propagate upwards. assert_equals: Main frame must not scroll vertically. expected 0 but got 3772
+FAIL Calling 'scrollIntoView()' inside a <iframe> with 'vertical-scroll none;'will not propagate upwards. assert_equals: Main frame must not scroll vertically. expected 0 but got 3851
 
--- /Volumes/Data/worker/Apple-iOS-16-Simulator-Release-arm64-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/feature-policy/reporting/picture-in-picture-reporting-expected.txt
+++ /Volumes/Data/worker/Apple-iOS-16-Simulator-Release-arm64-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/feature-policy/reporting/picture-in-picture-reporting-actual.txt
@@ -1,3 +1,3 @@
 
-FAIL Picture-in-Picture Report Format promise_rejects_dom: Picture-in-Picture should not be allowed in this document. function "function () { throw e }" threw object "InvalidStateError: The video element does not have a video track or it has not detected a video track yet." that is not a DOMException SecurityError: property "code" is equal to 11, expected 18
+FAIL Picture-in-Picture Report Format promise_rejects_dom: Picture-in-Picture should not be allowed in this document. function "function () { throw e }" threw object "NotSupportedError: The Picture-in-Picture mode is not supported." that is not a DOMException SecurityError: property "code" is equal to 9, expected 18

--- /Volumes/Data/worker/Apple-iPadOS-16-Simulator-Debug-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/feature-policy/feature-policy-frame-policy-allowed-for-some-override.https.sub-expected.txt
+++ /Volumes/Data/worker/Apple-iPadOS-16-Simulator-Debug-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/feature-policy/feature-policy-frame-policy-allowed-for-some-override.https.sub-actual.txt
@@ -1,3 +1,15 @@
+Blocked access to external URL https://www.localhost:9443/feature-policy/resources/feature-policy-allowedfeatures.html
+Blocked access to external URL https://www1.localhost:9443/feature-policy/resources/feature-policy-allowedfeatures.html
+Blocked access to external URL https://www.localhost:9443/feature-policy/resources/feature-policy-allowedfeatures.html
+Blocked access to external URL https://www1.localhost:9443/feature-policy/resources/feature-policy-allowedfeatures.html
+CONSOLE MESSAGE: TypeError: undefined is not an object (evaluating 'document.featurePolicy.allowedFeatures')
+Blocked access to external URL https://www.localhost:9443/feature-policy/resources/feature-policy-allowedfeatures.html
+Blocked access to external URL https://www1.localhost:9443/feature-policy/resources/feature-policy-allowedfeatures.html
+CONSOLE MESSAGE: TypeError: undefined is not an object (evaluating 'document.featurePolicy.allowedFeatures')
+Blocked access to external URL https://www.localhost:9443/feature-policy/resources/feature-policy-allowedfeatures.html
+Blocked access to external URL https://www1.localhost:9443/feature-policy/resources/feature-policy-allowedfeatures.html
+CONSOLE MESSAGE: TypeError: undefined is not an object (evaluating 'document.featurePolicy.allowedFeatures')
+CONSOLE MESSAGE: TypeError: undefined is not an object (evaluating 'document.featurePolicy.allowedFeatures')
 
 
 FAIL Test frame policy on same origin iframe with allow = "*" and allowfullscreen. undefined is not an object (evaluating 'frame_policy.allowedFeatures')

[Radar WebKit Bug Importer]

<rdar://problem/100719716>

[Hercules Hjalmarsson]

This issue can be bisected to 255047@main using command:

run-webkit-tests --iterations=2  --ios-simulator  imported/w3c/web-platform-tests/feature-policy/reporting/picture-in-picture-reporting.html

[EWS]

Test gardening commit 255089@main (eea3559482d2): <https://commits.webkit.org/255089@main>

Reviewed commits have been landed. Closing PR #4932 and removing active labels.

[Hercules Hjalmarsson]

Corrected history link:

https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Ffeature-policy%2Fexperimental-features%2Fvertical-scroll-scrollintoview.tentative.html&test=imported%2Fw3c%2Fweb-platform-tests%2Ffeature-policy%2Ffeature-policy-frame-policy-allowed-for-some-override.https.sub.html&test=imported%2Fw3c%2Fweb-platform-tests%2Ffeature-policy%2Freporting%2Fpicture-in-picture-reporting.html

[EWS]

Test gardening commit 256250@main (cd4a1c53cdbb): <https://commits.webkit.org/256250@main>

Reviewed commits have been landed. Closing PR #6056 and removing active labels.

[Hercules Hjalmarsson]

(In reply to EWS from comment #5)
> Test gardening commit 256250@main (cd4a1c53cdbb):
> <https://commits.webkit.org/256250@main>
> 
> Reviewed commits have been landed. Closing PR #6056 and removing active
> labels.

Re-baselined 2 out of 3 in above commit since last one is flakily failing. (imported/w3c/web-platform-tests/feature-policy/feature-policy-frame-policy-allowed-for-some-override.https.sub.html)

[Chris Dumez]

(In reply to Hercules Hjalmarsson from comment #6)
> (In reply to EWS from comment #5)
> > Test gardening commit 256250@main (cd4a1c53cdbb):
> > <https://commits.webkit.org/256250@main>
> > 
> > Reviewed commits have been landed. Closing PR #6056 and removing active
> > labels.
> 
> Re-baselined 2 out of 3 in above commit since last one is flakily failing.
> (imported/w3c/web-platform-tests/feature-policy/feature-policy-frame-policy-
> allowed-for-some-override.https.sub.html)

The only remaining failure is imported/w3c/web-platform-tests/feature-policy/feature-policy-frame-policy-
> allowed-for-some-override.https.sub.html and it is expected since the test relies on custom localhost domains which Cocoa ports don't support yet.