Bug 161912 – LayoutTest http/tests/security/cross-origin-cached-scripts-parallel.html is a flaky failure

• Component: New Bugs
• Status: RESOLVED FIXED

[Ryan Haddad]

LayoutTest http/tests/security/cross-origin-cached-scripts-parallel.html is a flaky failure

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fsecurity%2Fcross-origin-cached-scripts-parallel.html

--- /Volumes/Data/slave/yosemite-debug-tests-wk2/build/layout-test-results/http/tests/security/cross-origin-cached-scripts-parallel-expected.txt
+++ /Volumes/Data/slave/yosemite-debug-tests-wk2/build/layout-test-results/http/tests/security/cross-origin-cached-scripts-parallel-actual.txt
@@ -1,6 +1,6 @@
+CONSOLE MESSAGE: Origin http://localhost:8080 is not allowed by Access-Control-Allow-Origin.
 CONSOLE MESSAGE: Origin http://localhost:8080 is not allowed by Access-Control-Allow-Origin.
 CONSOLE MESSAGE: Cross-origin script load denied by Cross-Origin Resource Sharing policy.
-CONSOLE MESSAGE: Origin http://localhost:8080 is not allowed by Access-Control-Allow-Origin.
 CONSOLE MESSAGE: Cross-origin script load denied by Cross-Origin Resource Sharing policy.
 Tests source origin difference for cached resources.

[Ryan Haddad]

Marked test as flaky in https://trac.webkit.org/r205864 because it looks like this is an issue with the way the test is handling console messages.

Youenn, can you please take a look?

[youenn fablet]

(In reply to comment #1)
> Marked test as flaky in https://trac.webkit.org/r205864 because it looks
> like this is an issue with the way the test is handling console messages.
> 
> Youenn, can you please take a look?

preloads are sone in parallel and the CORS checks are done once each response arrives.
The test sorts the results but cannot sort the console.log since it is done within WebCore.
I will try to see how to change the test execution so as to minimise this case.

[youenn fablet]

Created attachment 288821 [details]
Patch

[youenn fablet]

Doing --iterations=100 confirmed the flakiness for the old test.
The new test is robust against --iterations=100.

[Alexey Proskuryakov]

Comment on attachment 288821 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=288821&action=review

> LayoutTests/http/tests/security/cross-origin-cached-scripts-parallel.html:40
> -var allow8000Script1 = "http://127.0.0.1:8000/security/resources/allow-if-origin.php?allowCache&origin=http%3A%2F%2Flocalhost%3A8000&name=notify-loaded.js&delay=100";
> -var allow8000Script2 = "http://127.0.0.1:8080/security/resources/allow-if-origin.php?allowCache&origin=http%3A%2F%2Flocalhost%3A8000&name=notify-loaded.js&delay=100";
> +var allow8000Script1 = "http://127.0.0.1:8000/security/resources/allow-if-origin.php?allowCache&origin=http%3A%2F%2Flocalhost%3A8000&name=notify-loaded.js&delay=1000";
> +var allow8000Script2 = "http://127.0.0.1:8080/security/resources/allow-if-origin.php?allowCache&origin=http%3A%2F%2Flocalhost%3A8000&name=notify-loaded.js&delay=1000";

In general, a delay of just one second is not sufficient for tests when running in parallel. They can well get flaky if there is something resource intensive happening, such as crash log generation.

Could you please confirm what happens when the test is run in parallel? For a laptop, this should do it, and a Mac Pro would need more child processes:

run-webkit-tests http/tests/security/cross-origin-cached-scripts-parallel.html --fully-parallel --iterations 1000 --child-processes=20

And maybe also with GuardMalloc (--guard-malloc).

[WebKit Commit Bot]

Comment on attachment 288821 [details]
Patch

Clearing flags on attachment: 288821

Committed r205908: <http://trac.webkit.org/changeset/205908>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.