Bug 197952 – Layout Test http/wpt/cache-storage/cache-quota-add.any.html is a flaky failure

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Shawn Roberts]

The following layout test is failing on iOS Simulator Release

http/wpt/cache-storage/cache-quota-add.any.html

Probable cause:

Test was added in https://trac.webkit.org/changeset/244918/webkit and is a very consistent failure on iOS Simulator Release.

Test is failing, but is a harness timeout

I cannot reproduce it locally. I have tried the test by itself with 5000 iterations multiple times. I have tried it in Guard malloc and Asan and still cannot replicate.
I have run it with the worker list from run #4135 and cannot get it to fail running that test list, even with 25 iterations. Test list from https://build.webkit.org/builders/Apple%20iOS%2012%20Simulator%20Release%20WK2%20%28Tests%29/builds/4135

Will add additional configuration attempts to reproduce in the Radar.

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Fwpt%2Fcache-storage%2Fcache-quota-add.any.html

Diff:

--- /Volumes/Data/slave/ios-simulator-12-release-tests-wk2/build/layout-test-results/http/wpt/cache-storage/cache-quota-add.any-expected.txt
+++ /Volumes/Data/slave/ios-simulator-12-release-tests-wk2/build/layout-test-results/http/wpt/cache-storage/cache-quota-add.any-actual.txt
@@ -1,6 +1,6 @@
-CONSOLE MESSAGE: Cache API operation failed: Quota exceeded
-CONSOLE MESSAGE: Cache API operation failed: Quota exceeded
 
-PASS Testing that cache.add checks against quota 
-PASS Testing that cache.addAll checks against quota 
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT Testing that cache.add checks against quota Test timed out
+NOTRUN Testing that cache.addAll checks against quota 
+

[Radar WebKit Bug Importer]

<rdar://problem/50859168>

[youenn fablet]

Created attachment 370049 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 370049 [details]
Patch

Clearing flags on attachment: 370049

Committed r245402: <https://trac.webkit.org/changeset/245402>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Shawn Roberts]

After changes in https://trac.webkit.org/changeset/245402 , it looks like the added slow flag worked on iOS. 

Test is failing with a harness timeout error, but the Diff is slightly different.

Still cannot reproduce locally.

Youenn do I just need to mark it slow there as well? Or did something else regress this?

Diff:

--- /Volumes/Data/slave/mojave-debug-tests-wk2/build/layout-test-results/http/wpt/cache-storage/cache-quota-add.any-expected.txt
+++ /Volumes/Data/slave/mojave-debug-tests-wk2/build/layout-test-results/http/wpt/cache-storage/cache-quota-add.any-actual.txt
@@ -1,6 +1,7 @@
-CONSOLE MESSAGE: Cache API operation failed: Quota exceeded
 CONSOLE MESSAGE: Cache API operation failed: Quota exceeded
 
+Harness Error (TIMEOUT), message = null
+
 PASS Testing that cache.add checks against quota 
-PASS Testing that cache.addAll checks against quota 
+TIMEOUT Testing that cache.addAll checks against quota Test timed out

[Shawn Roberts]

New Failure is on Mojave Release and Debug

[youenn fablet]

(In reply to Shawn Roberts from comment #6)
> New Failure is on Mojave Release and Debug

It seems there is a regression, not only this test is affected.

[youenn fablet]

Flakiness dashboard is good now.