The following layout test is failing on Sierra and High Sierra WK2 http/tests/security/bypassing-cors-checks-for-extension-urls.html Probable cause: Investigating further. The test just became flakey on 7/12/18 and around r233780. Investigating further but at this time I am not certain what revision started the flakiness. What I can gather is that the test is no longer reliably receiving two console messages at the start of the output. Flakiness Dashboard: https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fsecurity%2Fbypassing-cors-checks-for-extension-urls.html diff text: --- /Volumes/Data/slave/highsierra-release-tests-wk2/build/layout-test-results/http/tests/security/bypassing-cors-checks-for-extension-urls-expected.txt +++ /Volumes/Data/slave/highsierra-release-tests-wk2/build/layout-test-results/http/tests/security/bypassing-cors-checks-for-extension-urls-actual.txt @@ -1,5 +1,3 @@ -CONSOLE MESSAGE: line 19: Cross origin requests are only supported for HTTP. -CONSOLE MESSAGE: line 19: XMLHttpRequest cannot load safari-extension://test1 due to access control checks. This test ensures that XHR/Fetch will bypass CORS for user extension URLS in case the page is running user scripts.
We should find out what revision caused this since it happened so recently.
Strange thing while trying to reproduce. if you run the test with a test runner it will pass the first time then constantly fail all iterations after that. run it with 6 runners and it passes 6 times and fails all further iterations. Tested and reproduced on r233780 and r233750. going to test further back.
This may mean that the test is flaky because a preceding test sets some state, in the same way as this test breaks attempts to run it more than once in a single process. Definitely worth getting a repro case.
<rdar://problem/42306442>
*** This bug has been marked as a duplicate of bug 187541 ***
marked dup accidentally.
It was Noticed that this test: http/tests/security/blocked-on-redirect.html always seems to run directly before: http/tests/security/bypassing-cors-checks-for-extension-urls.html on the same test runner when it fails. If they are both run together repeatedly then they both will fail continuously. Example: 19:17:33.285 95669 worker/5 http/tests/security/blocked-on-redirect.html passed 19:17:33.444 95661 worker/1 http/tests/navigation/replacestate-base-legal.html passed 19:17:33.623 95661 worker/1 http/tests/navigation/replacestate-updates-referrer.html passed 19:17:33.910 95669 "ruby -I /Volumes/Data/slave/highsierra-debug-tests-wk2/build/Websites/bugs.webkit.org/PrettyPatch /Volumes/Data/slave/highsierra-debug-tests-wk2/build/Websites/bugs.webkit.org/PrettyPatch/prettify.rb /Volumes/Data/slave/highsierra-debug-tests-wk2/build/layout-test-results/http/tests/security/bypassing-cors-checks-for-extension-urls-diff.txt" took 0.26s 19:17:33.913 95617 [22595/50037] http/tests/security/bypassing-cors-checks-for-extension-urls.html failed unexpectedly (text diff) 19:17:33.912 95669 worker/5 http/tests/security/bypassing-cors-checks-for-extension-urls.html failed: 19:17:33.912 95669 worker/5 text diff
Marked test as flaky in https://trac.webkit.org/r234101
Update: After bisecting through the test runner this test ran on I found the test that triggers the failure of http/tests/security/bypassing-cors-checks-for-extension-urls.html. http/tests/contentextensions/injected-script-callback.html will cause this failure. Command: run-webkit-tests --root testbuild-235549 http/tests/contentextensions/injected-script-callback.html http/tests/security/bypassing-cors-checks-for-extension-urls.html --child-processes 1 --force --no-retry above command will %100 cause http/tests/security/bypassing-cors-checks-for-extension-urls.html to fail. http/tests/contentextensions/injected-script-callback.html was added in https://trac.webkit.org/changeset/233782/webkit which lines up with when http/tests/security/bypassing-cors-checks-for-extension-urls.html became flakey. additional validation: I skipped running http/tests/contentextensions/injected-script-callback.html in the test runner I was bisecting and no failure occurred with http/tests/security/bypassing-cors-checks-for-extension-urls.html
Created attachment 348696 [details] Patch
Comment on attachment 348696 [details] Patch Clearing flags on attachment: 348696 Committed r235602: <https://trac.webkit.org/changeset/235602>
All reviewed patches have been landed. Closing bug.