Bug 161912 - LayoutTest http/tests/security/cross-origin-cached-scripts-parallel.html is a flaky failure
Summary: LayoutTest http/tests/security/cross-origin-cached-scripts-parallel.html is a...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: youenn fablet
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-09-13 09:49 PDT by Ryan Haddad
Modified: 2016-09-14 09:42 PDT (History)
3 users (show)

See Also:


Attachments
Patch (7.26 KB, patch)
2016-09-14 08:50 PDT, youenn fablet
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ryan Haddad 2016-09-13 09:49:59 PDT
LayoutTest http/tests/security/cross-origin-cached-scripts-parallel.html is a flaky failure

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fsecurity%2Fcross-origin-cached-scripts-parallel.html

--- /Volumes/Data/slave/yosemite-debug-tests-wk2/build/layout-test-results/http/tests/security/cross-origin-cached-scripts-parallel-expected.txt
+++ /Volumes/Data/slave/yosemite-debug-tests-wk2/build/layout-test-results/http/tests/security/cross-origin-cached-scripts-parallel-actual.txt
@@ -1,6 +1,6 @@
+CONSOLE MESSAGE: Origin http://localhost:8080 is not allowed by Access-Control-Allow-Origin.
 CONSOLE MESSAGE: Origin http://localhost:8080 is not allowed by Access-Control-Allow-Origin.
 CONSOLE MESSAGE: Cross-origin script load denied by Cross-Origin Resource Sharing policy.
-CONSOLE MESSAGE: Origin http://localhost:8080 is not allowed by Access-Control-Allow-Origin.
 CONSOLE MESSAGE: Cross-origin script load denied by Cross-Origin Resource Sharing policy.
 Tests source origin difference for cached resources.
Comment 1 Ryan Haddad 2016-09-13 10:58:03 PDT
Marked test as flaky in https://trac.webkit.org/r205864 because it looks like this is an issue with the way the test is handling console messages.

Youenn, can you please take a look?
Comment 2 youenn fablet 2016-09-13 11:38:29 PDT
(In reply to comment #1)
> Marked test as flaky in https://trac.webkit.org/r205864 because it looks
> like this is an issue with the way the test is handling console messages.
> 
> Youenn, can you please take a look?

preloads are sone in parallel and the CORS checks are done once each response arrives.
The test sorts the results but cannot sort the console.log since it is done within WebCore.
I will try to see how to change the test execution so as to minimise this case.
Comment 3 youenn fablet 2016-09-14 08:50:55 PDT
Created attachment 288821 [details]
Patch
Comment 4 youenn fablet 2016-09-14 08:54:38 PDT
Doing --iterations=100 confirmed the flakiness for the old test.
The new test is robust against --iterations=100.
Comment 5 Alexey Proskuryakov 2016-09-14 09:18:11 PDT
Comment on attachment 288821 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=288821&action=review

> LayoutTests/http/tests/security/cross-origin-cached-scripts-parallel.html:40
> -var allow8000Script1 = "http://127.0.0.1:8000/security/resources/allow-if-origin.php?allowCache&origin=http%3A%2F%2Flocalhost%3A8000&name=notify-loaded.js&delay=100";
> -var allow8000Script2 = "http://127.0.0.1:8080/security/resources/allow-if-origin.php?allowCache&origin=http%3A%2F%2Flocalhost%3A8000&name=notify-loaded.js&delay=100";
> +var allow8000Script1 = "http://127.0.0.1:8000/security/resources/allow-if-origin.php?allowCache&origin=http%3A%2F%2Flocalhost%3A8000&name=notify-loaded.js&delay=1000";
> +var allow8000Script2 = "http://127.0.0.1:8080/security/resources/allow-if-origin.php?allowCache&origin=http%3A%2F%2Flocalhost%3A8000&name=notify-loaded.js&delay=1000";

In general, a delay of just one second is not sufficient for tests when running in parallel. They can well get flaky if there is something resource intensive happening, such as crash log generation.

Could you please confirm what happens when the test is run in parallel? For a laptop, this should do it, and a Mac Pro would need more child processes:

run-webkit-tests http/tests/security/cross-origin-cached-scripts-parallel.html --fully-parallel --iterations 1000 --child-processes=20

And maybe also with GuardMalloc (--guard-malloc).
Comment 6 WebKit Commit Bot 2016-09-14 09:42:20 PDT
Comment on attachment 288821 [details]
Patch

Clearing flags on attachment: 288821

Committed r205908: <http://trac.webkit.org/changeset/205908>
Comment 7 WebKit Commit Bot 2016-09-14 09:42:24 PDT
All reviewed patches have been landed.  Closing bug.