Bug 172454 – [WK2] Layout Test http/tests/navigation/statistics.html is flaky

• Status: NEW
• Component: Tools / Tests

[Matt Lewis]

The test http/tests/navigation/statistics.html is a flaky failure on WK2 Debug Sierra, El Capitan, and iOS Simulator.

The test recently started failing on all three platforms as of 5/19/2017

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fnavigation%2Fstatistics.html

https://build.webkit.org/results/Apple%20iOS%2010%20Simulator%20Debug%20WK2%20(Tests)/r217219%20(1539)/results.html
https://build.webkit.org/builders/Apple%20iOS%2010%20Simulator%20Debug%20WK2%20(Tests)/builds/1539

Diff:
--- /Volumes/Data/slave/ios-simulator-10-debug-tests-wk2/build/layout-test-results/http/tests/navigation/statistics-expected.txt
+++ /Volumes/Data/slave/ios-simulator-10-debug-tests-wk2/build/layout-test-results/http/tests/navigation/statistics-actual.txt
@@ -2,6 +2,29 @@
 main frame - has 1 onunload handler(s)
 PASS
 Dumping Resource Load Statistics:
+Statistics for 127.0.0.1:
+    hadUserInteraction: No
+    mostRecentUserInteraction: -1
+        grandfathered: No
+    topFrameHasBeenNavigatedToBefore: Yes
+    topFrameHasBeenRedirectedTo: 0
+    topFrameHasBeenRedirectedFrom: 0
+    topFrameInitialLoadCount: 0
+    topFrameHasBeenNavigatedTo: 1
+    topFrameHasBeenNavigatedFrom: 2
+    subframeHasBeenLoadedBefore: No
+    subframeHasBeenRedirectedTo: 0
+    subframeHasBeenRedirectedFrom: 0
+    subframeSubResourceCount: 0
+    subframeHasBeenNavigatedTo: 0
+    subframeHasBeenNavigatedFrom: 0
+    subresourceHasBeenRedirectedFrom: 0
+    subresourceHasBeenRedirectedTo: 0
+    subresourceHasBeenSubresourceCount: 0
+    subresourceHasBeenSubresourceCountDividedByTotalNumberOfOriginsVisited: 0
+    isPrevalentResource: No    dataRecordsRemoved: 0
+    isMarkedForCookiePartitioning: No
+

[Matt Lewis]

I was unable to reproduce the failure locally in Sierra Debug.

[Matt Lewis]

(In reply to Matt Lewis from comment #1)
> I was unable to reproduce the failure locally in Sierra Debug.

I managed to get the test to fail on Sierra Debug using this:

run-webkit-tests http/tests/navigation/ --debug --no-retry-failure -f

[Alexey Proskuryakov]

I wonder how this ever works:

        testRunner.notifyDone();
        log("Dumping Resource Load Statistics:");
        log(window.internals.resourceLoadStatisticsForOrigin("127.0.0.1"));

Brent, could you take a look at deflating the test? This is reproducible locally.

[Ryan Haddad]

Marked this test as flaky in http://trac.webkit.org/projects/webkit/changeset/217689

[Ryan Haddad]

(In reply to Ryan Haddad from comment #4)
> Marked this test as flaky in
> http://trac.webkit.org/projects/webkit/changeset/217689

This flakiness is not limited to debug. Removed debug only flag in http://trac.webkit.org/projects/webkit/changeset/217782

[Radar WebKit Bug Importer]

<rdar://problem/42793100>