Bug 195009 - [ Mac WK2] REGRESSION (r240046) Layout Test http/tests/security/cross-origin-local-storage* are flaky failures
Summary: [ Mac WK2] REGRESSION (r240046) Layout Test http/tests/security/cross-origin-...
Status: CLOSED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2019-02-25 10:54 PST by Shawn Roberts
Modified: 2019-03-21 13:31 PDT (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Shawn Roberts 2019-02-25 10:54:16 PST
The following layout tests are failing on Mac WK2

http/tests/security/cross-origin-local-storage-wk1.html
http/tests/security/cross-origin-local-storage-allowed.html
http/tests/security/cross-origin-local-storage.html

Probable cause:

All three tests started to fail with r240046.
Tests will fail when run individually as well as when run in a group

Reproducible with:

run-webkit-tests http/tests/security/cross-origin-local-storage* --iterations 500 -f

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fsecurity%2Fcross-origin-local-storage

Diff:

--- /Users/shawnroberts/Desktop/builds/t240046/layout-test-results/http/tests/security/cross-origin-local-storage-allowed-expected.txt
+++ /Users/shawnroberts/Desktop/builds/t240046/layout-test-results/http/tests/security/cross-origin-local-storage-allowed-actual.txt
@@ -1 +1 @@
-Got value: value
+No value
Comment 1 Radar WebKit Bug Importer 2019-02-25 10:55:38 PST
<rdar://problem/48369064>
Comment 2 Shawn Roberts 2019-03-08 13:57:08 PST
http/tests/security/cross-origin-local-storage-wk1.html
http/tests/security/cross-origin-local-storage-allowed.html
http/tests/security/cross-origin-local-storage.html

Additional repro steps:

Originally http/tests/security/cross-origin-local-storage-wk1.html appeared to have been a failure on the dashboard for the entire viewable history.

Both http/tests/security/cross-origin-local-storage-allowed.html http/tests/security/cross-origin-local-storage.html and failed for the first time together in revision 241662

Searching Trac, I could not find any recent changes to these files. Started my bisecting from the oldest revision I could spade. For passing revisions I used 10,000 iterations to make sure it was not just a very flaky failure in previous revisions. 

241719 failure 319 tests ran as expected, 5 didn't (1176 didn't run): failed unexpectedly (text diff)
241661 failure
241660 failure
240158 failure
240157 failure
240071 failure
240055 failure
240054 failure
240053 failure
240051 failure
240050 failure
240047 failure
240046 91 tests ran as expected, 5 didn't (1404 didn't run): failed unexpectedly (text diff)

240045 All Passing with 1000 iterations each
240039 Pass
240028 Pass
239984 Pass
239810 Pass
239407 Pass
238661 Pass
237157 Pass
234157 Pass
228049 pass
Comment 3 Chris Dumez 2019-03-20 16:27:17 PDT
Looks like they have not been flaky in a while. I believe this has been fixed.
Comment 4 Shawn Roberts 2019-03-21 11:35:51 PDT
Mac WK2 is good, looks happy on dashboard and not failing locally any longer.

There are some recent crashes on Debug WK1 for Mac but I cannot repro locally.

Closing out radar for this failure tho.