Bug 194602 – [ Mac WK2 ] Layout Test http/tests/storageAccess/request-and-grant-access-then-detach-should-not-have-access.html is a flaky failure

• Component: Tools / Tests
• Status: RESOLVED DUPLICATE

[Shawn Roberts]

The following layout test is flaky on Mac WK2

http/tests/storageAccess/request-and-grant-access-then-detach-should-not-have-access.html

Probable cause:

Is a flaky failure since r234440 when test was edited. 

Reproducible locally with:

run-webkit-tests t234440 http/tests/storageAccess/request-and-grant-access-then-detach-should-not-have-access.html --iterations 500 -f --exit-after-n-failures=1 --no-retry-failures

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2FstorageAccess%2Frequest-and-grant-access-then-detach-should-not-have-access.html

Diff:

--- /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/http/tests/storageAccess/request-and-grant-access-then-detach-should-not-have-access-expected.txt
+++ /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/http/tests/storageAccess/request-and-grant-access-then-detach-should-not-have-access-actual.txt
@@ -1,12 +1,5 @@
-Tests that a cross-origin iframe from a prevalent domain that is granted storage access and then is detached from the DOM does not have storage access.
+#PID UNRESPONSIVE - com.apple.WebKit.WebContent.Development (pid 37845)
+FAIL: Timed out waiting for notifyDone to be called
 
-On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
-
-
-PASS Storage access was granted. document.cookie == firstPartyCookie=value, cookies seen server-side == {"firstPartyCookie":"value"}
-PASS There is a storage access entry for localhost.
-PASS There is no storage access entry for localhost after iframe detach.
-PASS successfullyParsed is true

[Radar WebKit Bug Importer]

<rdar://problem/48045653>

[Shawn Roberts]

Additional repro steps

Searched trac for changes to http/tests/storageAccess/request-and-grant-access-then-detach-should-not-have-access.html

Found test was last edited on https://trac.webkit.org/changeset/234440/webkit

Used this revision as the bisecting point.

Started with newest available revision r242640 --iterations 500 -f --exit-after-n-failures=1 --no-retry-failures
Results : 2 tests ran as expected, 1 didn't (497 didn't run):
Regressions: Unexpected text-only failures (1)
  http/tests/storageAccess/request-and-grant-access-then-detach-should-not-have-access.html [ Failure ]


Ran same test configuration with revision 234440 --iterations 500 -f --exit-after-n-failures=1 --no-retry-failures
Results : 3 tests ran as expected, 1 didn't (496 didn't run):
Regressions: Unexpected text-only failures (1)
  http/tests/storageAccess/request-and-grant-access-then-detach-should-not-have-access.html [ Failure ]


Finally ran tests with revision 234439 --iterations 500 -f --exit-after-n-failures=50
Results : 
0 tests ran as expected, 50 didn't (450 didn't run):
Regressions: Unexpected timeouts (1)
  http/tests/storageAccess/request-and-grant-access-then-detach-should-not-have-access.html [ Timeout ]

[Shawn Roberts]

Correction, code changed in 234440. 

Test was originally added in https://trac.webkit.org/changeset/226235/webkit and has likely been a flaky failure prior to r234440

[Ryosuke Niwa]

*** This bug has been marked as a duplicate of bug 194164 ***