Bug 207583 – [ iOS wk2 ] http/wpt/beacon/cors/crossorigin-arraybufferview-no-preflight.html is failing.

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Jason Lawrence]

http/wpt/beacon/cors/crossorigin-arraybufferview-no-preflight.html

Description:
This test is flaky failing on iOS wk2.


History:
https://results.webkit.org/?suite=layout-tests&test=http%2Fwpt%2Fbeacon%2Fcors%2Fcrossorigin-arraybufferview-no-preflight.html&platform=ios

Diff:
--- /Volumes/Data/slave/ios-simulator-13-release-tests-wk2/build/layout-test-results/http/wpt/beacon/cors/crossorigin-arraybufferview-no-preflight-expected.txt
+++ /Volumes/Data/slave/ios-simulator-13-release-tests-wk2/build/layout-test-results/http/wpt/beacon/cors/crossorigin-arraybufferview-no-preflight-actual.txt
@@ -1,4 +1,4 @@
 
 PASS Beacon load should not be sent if server is not trusted. 
-PASS Should send beacon with no CORS preflight 
+FAIL Should send beacon with no CORS preflight assert_equals: Received beacon expected 1 but got 0

[Radar WebKit Bug Importer]

<rdar://problem/59362947>

[Jason Lawrence]

I marked this test as failing while this issue is investigated.
https://trac.webkit.org/changeset/256370/webkit

[Jason Lawrence]

I see the same diff generated from a test on r252356 here: 
https://build.webkit.org/builders/Apple%20iPadOS%2013%20Simulator%20Debug%20WK2%20%28Tests%29/builds/553

It looks like the issue may regress back to r250649, but I cannot recover a diff.

[Jason Lawrence]

I am able to reproduce this issue with r256586 using the command below. 
run-webkit-tests http/wpt/beacon/cors/crossorigin-arraybufferview-no-preflight.html --force -f --iterations 5000 --exit-after-n-failures 5 --ios-simulator

[3/5000] http/wpt/beacon/cors/crossorigin-arraybufferview-no-preflight.html failed unexpectedly (text diff)
[3/5000] http/wpt/beacon/cors/crossorigin-arraybufferview-no-preflight.html failed unexpectedly (text diff)
[3/5000] http/wpt/beacon/cors/crossorigin-arraybufferview-no-preflight.html failed unexpectedly (text diff)
[6/5000] http/wpt/beacon/cors/crossorigin-arraybufferview-no-preflight.html failed unexpectedly (text diff)
[7/5000] http/wpt/beacon/cors/crossorigin-arraybufferview-no-preflight.html failed unexpectedly (text diff)
Exiting early after 5 failures. 5 tests run.

[Ryan Haddad]

This test went from being a flaky failure to a consistent failure somewhere in this range https://trac.webkit.org/log/webkit/?action=stop_on_copy&mode=stop_on_copy&rev=256373+&stop_rev=256344+&limit=100&verbose=on

--- /Volumes/Data/slave/ios-simulator-13-release-tests-wk2/build/layout-test-results/http/wpt/beacon/cors/crossorigin-arraybufferview-no-preflight-expected.txt
+++ /Volumes/Data/slave/ios-simulator-13-release-tests-wk2/build/layout-test-results/http/wpt/beacon/cors/crossorigin-arraybufferview-no-preflight-actual.txt
@@ -1,3 +1,6 @@
+CONSOLE MESSAGE: Beacon API cannot load https://127.0.0.1:9443/WebKit/beacon/resources/beacon-preflight.py?allowCors=1&cmd=put&id=ca33c073-1cf0-41e5-bbd4-55969fb1a6c7. The certificate for this server is invalid. You might be connecting to a server that is pretending to be “127.0.0.1” which could put your confidential information at risk.
+CONSOLE MESSAGE: Origin http://localhost:8800 is not allowed by Access-Control-Allow-Origin.
+CONSOLE MESSAGE: Beacon API cannot load https://127.0.0.1:9443/WebKit/beacon/resources/beacon-preflight.py?allowCors=1&cmd=put&id=bca6775f-2c84-4faf-94b5-454ca0b7340d. Origin http://localhost:8800 is not allowed by Access-Control-Allow-Origin.
 
 PASS Beacon load should not be sent if server is not trusted. 
 PASS Should send beacon with no CORS preflight

[youenn fablet]

Created attachment 399001 [details]
Patch

[Geoffrey Garen]

Comment on attachment 399001 [details]
Patch

r=me

[Chris Dumez]

Comment on attachment 399001 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=399001&action=review

> LayoutTests/ChangeLog:9
> +        Use in-test DumpJSConsoleLogInStdErr instead of TestExpectations to make sure this option is set for iOS.

If TestExpectations in TestExpectations does not apply on iOS, isn't it a bug in itself?

[EWS]

Committed r261483: <https://trac.webkit.org/changeset/261483>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 399001 [details].

[youenn fablet]

(In reply to Chris Dumez from comment #8)
> Comment on attachment 399001 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=399001&action=review
> 
> > LayoutTests/ChangeLog:9
> > +        Use in-test DumpJSConsoleLogInStdErr instead of TestExpectations to make sure this option is set for iOS.
> 
> If TestExpectations in TestExpectations does not apply on iOS, isn't it a
> bug in itself?

The test has two expectations (one in iOS and one generic, only the latter had DumpJSConsoleLogInStdErr).
It seems simpler to move to in-test than to try changing webkitpy infra.

[Jason Lawrence]

I have removed the expectations because this issue appears to be resolved.
https://trac.webkit.org/changeset/262163/webkit