LayoutTest http/tests/security/cross-origin-cached-scripts-parallel.html is a flaky failure https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fsecurity%2Fcross-origin-cached-scripts-parallel.html --- /Volumes/Data/slave/yosemite-debug-tests-wk2/build/layout-test-results/http/tests/security/cross-origin-cached-scripts-parallel-expected.txt +++ /Volumes/Data/slave/yosemite-debug-tests-wk2/build/layout-test-results/http/tests/security/cross-origin-cached-scripts-parallel-actual.txt @@ -1,6 +1,6 @@ +CONSOLE MESSAGE: Origin http://localhost:8080 is not allowed by Access-Control-Allow-Origin. CONSOLE MESSAGE: Origin http://localhost:8080 is not allowed by Access-Control-Allow-Origin. CONSOLE MESSAGE: Cross-origin script load denied by Cross-Origin Resource Sharing policy. -CONSOLE MESSAGE: Origin http://localhost:8080 is not allowed by Access-Control-Allow-Origin. CONSOLE MESSAGE: Cross-origin script load denied by Cross-Origin Resource Sharing policy. Tests source origin difference for cached resources.
Marked test as flaky in https://trac.webkit.org/r205864 because it looks like this is an issue with the way the test is handling console messages. Youenn, can you please take a look?
(In reply to comment #1) > Marked test as flaky in https://trac.webkit.org/r205864 because it looks > like this is an issue with the way the test is handling console messages. > > Youenn, can you please take a look? preloads are sone in parallel and the CORS checks are done once each response arrives. The test sorts the results but cannot sort the console.log since it is done within WebCore. I will try to see how to change the test execution so as to minimise this case.
Created attachment 288821 [details] Patch
Doing --iterations=100 confirmed the flakiness for the old test. The new test is robust against --iterations=100.
Comment on attachment 288821 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=288821&action=review > LayoutTests/http/tests/security/cross-origin-cached-scripts-parallel.html:40 > -var allow8000Script1 = "http://127.0.0.1:8000/security/resources/allow-if-origin.php?allowCache&origin=http%3A%2F%2Flocalhost%3A8000&name=notify-loaded.js&delay=100"; > -var allow8000Script2 = "http://127.0.0.1:8080/security/resources/allow-if-origin.php?allowCache&origin=http%3A%2F%2Flocalhost%3A8000&name=notify-loaded.js&delay=100"; > +var allow8000Script1 = "http://127.0.0.1:8000/security/resources/allow-if-origin.php?allowCache&origin=http%3A%2F%2Flocalhost%3A8000&name=notify-loaded.js&delay=1000"; > +var allow8000Script2 = "http://127.0.0.1:8080/security/resources/allow-if-origin.php?allowCache&origin=http%3A%2F%2Flocalhost%3A8000&name=notify-loaded.js&delay=1000"; In general, a delay of just one second is not sufficient for tests when running in parallel. They can well get flaky if there is something resource intensive happening, such as crash log generation. Could you please confirm what happens when the test is run in parallel? For a laptop, this should do it, and a Mac Pro would need more child processes: run-webkit-tests http/tests/security/cross-origin-cached-scripts-parallel.html --fully-parallel --iterations 1000 --child-processes=20 And maybe also with GuardMalloc (--guard-malloc).
Comment on attachment 288821 [details] Patch Clearing flags on attachment: 288821 Committed r205908: <http://trac.webkit.org/changeset/205908>
All reviewed patches have been landed. Closing bug.