Bug 168136 – LayoutTest performance-api/performance-observer-periodic.html is a flaky failure

• Status: NEW
• Component: New Bugs

[Ryan Haddad]

LayoutTest performance-api/performance-observer-periodic.html is a flaky failure

https://build.webkit.org/results/Apple%20El%20Capitan%20Debug%20WK2%20(Tests)/r212115%20(10958)/results.html

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=performance-api%2Fperformance-observer-periodic.html

--- /Volumes/Data/slave/elcapitan-debug-tests-wk2/build/layout-test-results/performance-api/performance-observer-periodic-expected.txt
+++ /Volumes/Data/slave/elcapitan-debug-tests-wk2/build/layout-test-results/performance-api/performance-observer-periodic-actual.txt
@@ -5,9 +5,6 @@
 
 PerformanceObserver callback fired: 1 entries
 PASS mark1
-PerformanceObserver callback fired: 2 entries
-PASS mark2
-PASS mark3
 PerformanceObserver callback fired: 1 entries
 PASS mark7
 PASS successfullyParsed is true

[Joseph Pecoraro]

Okay. So this can happen if the PerformanceObserver task didn't get to run between the 50ms and 100ms timeouts. Those timeouts could instead move into the PerformanceObserver...

[Ryan Haddad]

Marked test as flaky in http://trac.webkit.org/projects/webkit/changeset/213380

[Matt Lewis]

This test has started to flake on iOS 11 as well.

First failure:
https://build.webkit.org/results/Apple%20iOS%2011%20Simulator%20Debug%20WK2%20(Tests)/r222560%20(120)/results.html
https://build.webkit.org/builders/Apple%20iOS%2011%20Simulator%20Debug%20WK2%20(Tests)/builds/120

Diff:
--- /Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/layout-test-results/performance-api/performance-observer-periodic-expected.txt
+++ /Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/layout-test-results/performance-api/performance-observer-periodic-actual.txt
@@ -5,9 +5,6 @@
 
 PerformanceObserver callback fired: 1 entries
 PASS mark1
-PerformanceObserver callback fired: 2 entries
-PASS mark2
-PASS mark3
 PerformanceObserver callback fired: 1 entries
 PASS mark7
 PASS successfullyParsed is true

[Shawn Roberts]

Test is now a flaky failure on iOS 12 Simulator as well.

Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=performance-api%2Fperformance-observer-periodic.html

First appearance was around r243337 but I was able to reproduce it with r242010 so it appears it's been flaky for as long as the other platforms. Gets about 1 failure in 1000 iterations, highly flaky. 

Reproducible with :

run-webkit-tests performance-api/performance-observer-periodic.html --ios-simulator --iterations 1000 -f

--- /Volumes/Data/slave/ios-simulator-12-release-tests-wk2/build/layout-test-results/http/tests/cache-storage/cache-records-persistency.https-expected.txt
+++ /Volumes/Data/slave/ios-simulator-12-release-tests-wk2/build/layout-test-results/http/tests/cache-storage/cache-records-persistency.https-actual.txt
@@ -1,9 +1,7 @@
 
 PASS Cleaning existing caches 
-PASS Cache.put called twice with matching Requests - keys should remain the same - testing persistency 
+FAIL Cache.put called twice with matching Requests - keys should remain the same - testing persistency promise_test: Unhandled rejection with value: object "TypeError: undefined is not an object (evaluating 'result.text')"
 PASS Cleaning created caches 
-PASS Checking persistency of cache keys 
-PASS Checking persistency of response 
-PASS Checking persistency of response body 
-PASS Testing overriding of an existing cached record that is not in memory 
+FAIL Checking persistency of cache keys assert_equals: expected 1 but got 0
+FAIL Checking persistency of response assert_equals: Cache.put should store the response persistently expected "[object Response]" but got "[object Undefined]"

[Shawn Roberts]

Linked the wrong diff above.

--- /Volumes/Data/slave/ios-simulator-12-release-tests-wk2/build/layout-test-results/performance-api/performance-observer-periodic-expected.txt
+++ /Volumes/Data/slave/ios-simulator-12-release-tests-wk2/build/layout-test-results/performance-api/performance-observer-periodic-actual.txt
@@ -5,9 +5,6 @@
 
 PerformanceObserver callback fired: 1 entries
 PASS mark1
-PerformanceObserver callback fired: 2 entries
-PASS mark2
-PASS mark3
 PerformanceObserver callback fired: 1 entries
 PASS mark7
 PASS successfullyParsed is true