Bug 195385 - [ iOS Simulator ] REGRESSION (r237702) Layout Test http/tests/security/xss-DENIED-script-inject-into-inactive-window.html is a flaky failure
Summary: [ iOS Simulator ] REGRESSION (r237702) Layout Test http/tests/security/xss-DE...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Chris Dumez
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2019-03-06 16:15 PST by Shawn Roberts
Modified: 2019-03-27 12:56 PDT (History)
9 users (show)

See Also:


Attachments
Patch (2.65 KB, patch)
2019-03-27 12:15 PDT, Chris Dumez
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Shawn Roberts 2019-03-06 16:15:53 PST
The following layout test is failing on iOS Simulator

http/tests/security/xss-DENIED-script-inject-into-inactive-window.html

Probable cause:

Test has been a flaky failure since it was changed in r237702. Fails 100% of the time with any previous build.

Reproduced with:

run-webkit-tests http/tests/security/xss-DENIED-script-inject-into-inactive-window.html --iterations 100 -f --ios-simulator --child-processes 4

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fsecurity%2Fxss-DENIED-script-inject-into-inactive-window.html

Diff:

--- /Volumes/Data/slave/ios-simulator-12-release-tests-wk2/build/layout-test-results/http/tests/security/xss-DENIED-script-inject-into-inactive-window-expected.txt
+++ /Volumes/Data/slave/ios-simulator-12-release-tests-wk2/build/layout-test-results/http/tests/security/xss-DENIED-script-inject-into-inactive-window-actual.txt
@@ -1,3 +1,4 @@
+ALERT: XSS
 Check the child window. The test FAILED if you see a JavaScript alert() dialog. Otherwise, it PASSED.
Comment 1 Radar WebKit Bug Importer 2019-03-06 16:18:42 PST
<rdar://problem/48657425>
Comment 2 Shawn Roberts 2019-03-08 10:20:55 PST
Additional repro information:

Found test was a flaky failure on dashboard through its available history on iOS Simulator.

Searched on Trac and found test was modified in r237702 and used this as a bisecting point.

Re-tested without extra child processes using command : run-webkit-tests http/tests/security/xss-DENIED-script-inject-into-inactive-window.html --iterations 500 -f --ios-simulator --exit-after-n-failures=5 --no-retry-failures

242568 5 out of 234 failed unexpectedly (text diff)
237702 3 out of 500 failed unexpectedly (text diff)
237701 Changed test parameters to --exit-after-n-failures=25, 25 out of 25 Unexpected timeouts (1) [ Timeout ]
Comment 3 Chris Dumez 2019-03-27 12:15:50 PDT
Created attachment 366090 [details]
Patch
Comment 4 WebKit Commit Bot 2019-03-27 12:56:07 PDT
Comment on attachment 366090 [details]
Patch

Clearing flags on attachment: 366090

Committed r243558: <https://trac.webkit.org/changeset/243558>
Comment 5 WebKit Commit Bot 2019-03-27 12:56:09 PDT
All reviewed patches have been landed.  Closing bug.