[Catalina Release wk1] imported/w3c/web-platform-tests/html/browsers/origin/cross-origin-objects/cross-origin-objects.html is a flaky failure

• Component: DOM
• Status: RESOLVED FIXED

[Eric Hutchison]

imported/w3c/web-platform-tests/html/browsers/origin/cross-origin-objects/cross-origin-objects.html

is a flaky failure on Catalina Release wk1.

History: https://results.webkit.org/?suite=layout-tests&test=imported/w3c/web-platform-tests/html/browsers/origin/cross-origin-objects/cross-origin-objects.html

Results: https://ews-build.webkit.org/#/builders/57/builds/16285; https://ews-build.s3-us-west-2.amazonaws.com/macOS-Catalina-Release-WK1-Tests-EWS/r439084-16285/results.html

Diff: 
--- /Volumes/Data/worker/macOS-Catalina-Release-WK1-Tests-EWS/build/layout-test-results/imported/w3c/web-platform-tests/html/browsers/origin/cross-origin-objects/cross-origin-objects-expected.txt
+++ /Volumes/Data/worker/macOS-Catalina-Release-WK1-Tests-EWS/build/layout-test-results/imported/w3c/web-platform-tests/html/browsers/origin/cross-origin-objects/cross-origin-objects-actual.txt
@@ -1,3 +1,5 @@
+
+Harness Error (TIMEOUT), message = null
 
 PASS Basic sanity-checking (cross-origin)
 PASS Basic sanity-checking (same-origin + document.domain)
@@ -56,61 +58,4 @@
 PASS [[GetOwnProperty]] - Subframes should be visible cross-origin only if their names don't match the names of cross-origin-exposed IDL properties (cross-origin)
 PASS [[GetOwnProperty]] - Subframes should be visible cross-origin only if their names don't match the names of cross-origin-exposed IDL properties (same-origin + document.domain)
 PASS [[GetOwnProperty]] - Subframes should be visible cross-origin only if their names don't match the names of cross-origin-exposed IDL properties (cross-site)
-PASS [[GetOwnProperty]] - Should be able to get a property descriptor for an indexed property only if it corresponds to a child window. (cross-origin)
-PASS [[GetOwnProperty]] - Should be able to get a property descriptor for an indexed property only if it corresponds to a child window. (same-origin + document.domain)
-PASS [[GetOwnProperty]] - Should be able to get a property descriptor for an indexed property only if it corresponds to a child window. (cross-site)
-PASS [[Delete]] Should throw on cross-origin objects (cross-origin)
-PASS [[Delete]] Should throw on cross-origin objects (same-origin + document.domain)
-PASS [[Delete]] Should throw on cross-origin objects (cross-site)
-PASS [[DefineOwnProperty]] Should throw for cross-origin objects (cross-origin)
-PASS [[DefineOwnProperty]] Should throw for cross-origin objects (same-origin + document.domain)
-PASS [[DefineOwnProperty]] Should throw for cross-origin objects (cross-site)
-PASS Can only enumerate safelisted enumerable properties (cross-origin)
-PASS Can only enumerate safelisted enumerable properties (same-origin + document.domain)
-PASS Can only enumerate safelisted enumerable properties (cross-site)
-PASS [[OwnPropertyKeys]] should return all properties from cross-origin objects (cross-origin)
-PASS [[OwnPropertyKeys]] should return all properties from cross-origin objects (same-origin + document.domain)
-PASS [[OwnPropertyKeys]] should return all properties from cross-origin objects (cross-site)
-PASS [[OwnPropertyKeys]] should return the right symbol-named properties for cross-origin objects (cross-origin)
-PASS [[OwnPropertyKeys]] should return the right symbol-named properties for cross-origin objects (same-origin + document.domain)
-PASS [[OwnPropertyKeys]] should return the right symbol-named properties for cross-origin objects (cross-site)
-PASS [[OwnPropertyKeys]] should place the symbols after the property names after the subframe indices (cross-origin)
-PASS [[OwnPropertyKeys]] should place the symbols after the property names after the subframe indices (same-origin + document.domain)
-PASS [[OwnPropertyKeys]] should place the symbols after the property names after the subframe indices (cross-site)
-PASS [[OwnPropertyKeys]] should not reorder where 'then' appears if it's a named subframe, nor add another copy of 'then' (cross-origin)
-PASS [[OwnPropertyKeys]] should not reorder where 'then' appears if it's a named subframe, nor add another copy of 'then' (same-origin + document.domain)
-PASS [[OwnPropertyKeys]] should not reorder where 'then' appears if it's a named subframe, nor add another copy of 'then' (cross-site)
-PASS A and B jointly observe the same identity for cross-origin Window and Location (cross-origin)
-PASS A and B jointly observe the same identity for cross-origin Window and Location (same-origin + document.domain)
-PASS A and B jointly observe the same identity for cross-origin Window and Location (cross-site)
-PASS Cross-origin functions get local Function.prototype (cross-origin)
-PASS Cross-origin functions get local Function.prototype (same-origin + document.domain)
-PASS Cross-origin functions get local Function.prototype (cross-site)
-PASS Cross-origin Window accessors get local Function.prototype (cross-origin)
-PASS Cross-origin Window accessors get local Function.prototype (same-origin + document.domain)
-PASS Cross-origin Window accessors get local Function.prototype (cross-site)
-PASS Same-origin observers get different functions for cross-origin objects (cross-origin)
-PASS Same-origin observers get different functions for cross-origin objects (same-origin + document.domain)
-PASS Same-origin observers get different functions for cross-origin objects (cross-site)
-PASS Same-origin observers get different accessors for cross-origin Window (cross-origin)
-PASS Same-origin observers get different accessors for cross-origin Window (same-origin + document.domain)
-PASS Same-origin observers get different accessors for cross-origin Window (cross-site)
-PASS Same-origin observers get different accessors for cross-origin Location (cross-origin)
-PASS Same-origin observers get different accessors for cross-origin Location (same-origin + document.domain)
-PASS Same-origin observers get different accessors for cross-origin Location (cross-site)
-PASS {}.toString.call() does the right thing on cross-origin objects (cross-origin)
-PASS {}.toString.call() does the right thing on cross-origin objects (same-origin + document.domain)
-PASS {}.toString.call() does the right thing on cross-origin objects (cross-site)
-PASS Resolving a promise with a cross-origin window without a 'then' subframe should work (cross-origin)
-PASS Resolving a promise with a cross-origin window without a 'then' subframe should work (same-origin + document.domain)
-PASS Resolving a promise with a cross-origin window without a 'then' subframe should work (cross-site)
-PASS Resolving a promise with a cross-origin window with a 'then' subframe should work (cross-origin)
-PASS Resolving a promise with a cross-origin window with a 'then' subframe should work (same-origin + document.domain)
-PASS Resolving a promise with a cross-origin window with a 'then' subframe should work (cross-site)
-PASS Resolving a promise with a cross-origin location should work (cross-origin)
-PASS Resolving a promise with a cross-origin location should work (same-origin + document.domain)
-PASS Resolving a promise with a cross-origin location should work (cross-site)
-PASS LegacyLenientThis behavior (cross-origin)
-PASS LegacyLenientThis behavior (same-origin + document.domain)
-PASS LegacyLenientThis behavior (cross-site)

STDIO: 

worker/17 worker/17 imported/w3c/web-platform-tests/html/browsers/origin/cross-origin-objects/cross-origin-objects.html output stderr lines:
13:32:04.607 8981 worker/17   CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains, and ports must match.
13:32:04.608 8981 worker/17   CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains, and ports must match.
13:32:04.608 8981 worker/17   CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains, and ports must match.
13:32:04.608 8981 worker/17   CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://localhost:8800". The frame being accessed set "document.domain" to "localhost", but the frame requesting access did not. Both must set "document.domain" to the same value to allow access.
13:32:04.609 8981 worker/17   CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://localhost:8800". The frame being accessed set "document.domain" to "localhost", but the frame requesting access did not. Both must set "document.domain" to the same value to allow access.
13:32:04.609 8981 worker/17   CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://localhost:8800". The frame being accessed set "document.domain" to "localhost", but the frame requesting access did not. Both must set "document.domain" to the same value to allow access.
13:32:04.609 8981 worker/17   CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains, and ports must match.
13:32:04.610 8981 worker/17   CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains, and ports must match.
13:32:04.610 8981 worker/17   CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains, and ports must match.
13:32:04.610 8981 worker/17   CONSOLE MESSAGE: Unable to post message to http://does-not.exist. Recipient has origin http://127.0.0.1:8800.
13:32:04.611 8981 worker/17   
13:32:04.611 8981 worker/17   CONSOLE MESSAGE: Unable to post message to http://localhost:8800. Recipient has origin http://127.0.0.1:8800.
13:32:04.611 8981 worker/17   
13:32:04.611 8981 worker/17   CONSOLE MESSAGE: Unable to post message to http://does-not.exist. Recipient has origin http://127.0.0.1:8800.
13:32:04.612 8981 worker/17   
13:32:04.612 8981 worker/17   CONSOLE MESSAGE: Unable to post message to http://localhost:8800. Recipient has origin http://127.0.0.1:8800.
13:32:04.612 8981 worker/17   
13:32:04.612 8981 worker/17   CONSOLE MESSAGE: Unable to post message to http://does-not.exist. Recipient has origin http://localhost:8800.
13:32:04.613 8981 worker/17   

Unable to attempt repro locally due to lack of equipment.

[Radar WebKit Bug Importer]

<rdar://problem/83469625>

[Eric Hutchison]

Updated test expectations at https://trac.webkit.org/changeset/283018/webkit

[Chris Dumez]

The diff I see on the bots is:
--- /Volumes/Data/worker/catalina-release-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/html/browsers/origin/cross-origin-objects/cross-origin-objects-expected.txt
+++ /Volumes/Data/worker/catalina-release-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/html/browsers/origin/cross-origin-objects/cross-origin-objects-actual.txt
@@ -1,3 +1,22 @@
+CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains, and ports must match.
+CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains, and ports must match.
+CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains, and ports must match.
+CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://localhost:8800". The frame being accessed set "document.domain" to "localhost", but the frame requesting access did not. Both must set "document.domain" to the same value to allow access.
+CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://localhost:8800". The frame being accessed set "document.domain" to "localhost", but the frame requesting access did not. Both must set "document.domain" to the same value to allow access.
+CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://localhost:8800". The frame being accessed set "document.domain" to "localhost", but the frame requesting access did not. Both must set "document.domain" to the same value to allow access.
+CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains, and ports must match.
+CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains, and ports must match.
+CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains, and ports must match.
+CONSOLE MESSAGE: Unable to post message to http://does-not.exist. Recipient has origin http://127.0.0.1:8800.
+
+CONSOLE MESSAGE: Unable to post message to http://localhost:8800. Recipient has origin http://127.0.0.1:8800.
+
+CONSOLE MESSAGE: Unable to post message to http://does-not.exist. Recipient has origin http://127.0.0.1:8800.
+
+CONSOLE MESSAGE: Unable to post message to http://localhost:8800. Recipient has origin http://127.0.0.1:8800.
+
+CONSOLE MESSAGE: Unable to post message to http://does-not.exist. Recipient has origin http://localhost:8800.
+
 
 PASS Basic sanity-checking (cross-origin)
 PASS Basic sanity-checking (same-origin + document.domain)


It is fishy because it is only failing on this one bot AND we have the following line in TestExpectations:
imported/w3c/web-platform-tests/html/browsers/origin/cross-origin-objects/cross-origin-objects.html [ DumpJSConsoleLogInStdErr ]

So somehow, the [ DumpJSConsoleLogInStdErr ] in TestExpectations is ignored and the test output still contains console messages on this particular bot?

[Chris Dumez]

(In reply to Chris Dumez from comment #3)
> The diff I see on the bots is:
> ---
> /Volumes/Data/worker/catalina-release-tests-wk1/build/layout-test-results/
> imported/w3c/web-platform-tests/html/browsers/origin/cross-origin-objects/
> cross-origin-objects-expected.txt
> +++
> /Volumes/Data/worker/catalina-release-tests-wk1/build/layout-test-results/
> imported/w3c/web-platform-tests/html/browsers/origin/cross-origin-objects/
> cross-origin-objects-actual.txt
> @@ -1,3 +1,22 @@
> +CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from
> accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains,
> and ports must match.
> +CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from
> accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains,
> and ports must match.
> +CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from
> accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains,
> and ports must match.
> +CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from
> accessing a frame with origin "http://localhost:8800". The frame being
> accessed set "document.domain" to "localhost", but the frame requesting
> access did not. Both must set "document.domain" to the same value to allow
> access.
> +CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from
> accessing a frame with origin "http://localhost:8800". The frame being
> accessed set "document.domain" to "localhost", but the frame requesting
> access did not. Both must set "document.domain" to the same value to allow
> access.
> +CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from
> accessing a frame with origin "http://localhost:8800". The frame being
> accessed set "document.domain" to "localhost", but the frame requesting
> access did not. Both must set "document.domain" to the same value to allow
> access.
> +CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from
> accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains,
> and ports must match.
> +CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from
> accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains,
> and ports must match.
> +CONSOLE MESSAGE: Blocked a frame with origin "http://localhost:8800" from
> accessing a frame with origin "http://127.0.0.1:8800". Protocols, domains,
> and ports must match.
> +CONSOLE MESSAGE: Unable to post message to http://does-not.exist. Recipient
> has origin http://127.0.0.1:8800.
> +
> +CONSOLE MESSAGE: Unable to post message to http://localhost:8800. Recipient
> has origin http://127.0.0.1:8800.
> +
> +CONSOLE MESSAGE: Unable to post message to http://does-not.exist. Recipient
> has origin http://127.0.0.1:8800.
> +
> +CONSOLE MESSAGE: Unable to post message to http://localhost:8800. Recipient
> has origin http://127.0.0.1:8800.
> +
> +CONSOLE MESSAGE: Unable to post message to http://does-not.exist. Recipient
> has origin http://localhost:8800.
> +
>  
>  PASS Basic sanity-checking (cross-origin)
>  PASS Basic sanity-checking (same-origin + document.domain)
> 
> 
> It is fishy because it is only failing on this one bot AND we have the
> following line in TestExpectations:
> imported/w3c/web-platform-tests/html/browsers/origin/cross-origin-objects/
> cross-origin-objects.html [ DumpJSConsoleLogInStdErr ]
> 
> So somehow, the [ DumpJSConsoleLogInStdErr ] in TestExpectations is ignored
> and the test output still contains console messages on this particular bot?

Or maybe this diff is simply the result of Eric marking this test as flaky here:
LayoutTests/platform/mac-wk1/TestExpectations:webkit.org/b/230729 [ Catalina Release ] imported/w3c/web-platform-tests/html/browsers/origin/cross-origin-objects/cross-origin-objects.html [ Pass Failure ]

Maybe [ Pass Failure ] in mac-wk1's TestExpectations overrides [ DumpJSConsoleLogInStdErr ] in the global TestExpectations.

[Chris Dumez]

The test in general is slow and the output from https://bugs.webkit.org/show_bug.cgi?id=230729# seems to indicate that the time is sometimes a little too slow and times out. We may need to mark this test as [ Slow ].

[Chris Dumez]

Committed r290891 (?): <https://commits.webkit.org/r290891>