Bug 194525 – Layout Test http/wpt/fetch/response-opaque-clone.html is sometimes timing out on iOS simulator

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Shawn Roberts]

The following layout test is flaky on Mac WK2

http/wpt/fetch/response-opaque-clone.html

Probable cause:

Test appears to have been changed with revision 240158. Is a flaky failure with several versions tested from 240158 to 241282

Reproduced with:

run-webkit-tests t241282 http/wpt/fetch/response-opaque-clone.html --iterations 2500 -f

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Fwpt%2Ffetch%2Fresponse-opaque-clone.html

Diff:

--- /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/http/wpt/fetch/response-opaque-clone-expected.txt
+++ /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/http/wpt/fetch/response-opaque-clone-actual.txt
@@ -1,3 +1,5 @@
+#PID UNRESPONSIVE - com.apple.WebKit.WebContent.Development (pid 38099)
+FAIL: Timed out waiting for notifyDone to be called
 
-PASS Check opaque response can be cloned and cached correctly 
-
+#EOF
+#EOF

[Radar WebKit Bug Importer]

<rdar://problem/48012229>

[Alexey Proskuryakov]

> run-webkit-tests t241282 http/wpt/fetch/response-opaque-clone.html --iterations 2500 -f

Just to double check, does this not reproduce with r240157? I want to make sure that the repro case captures what regressed, not some other issue.

[Shawn Roberts]

(In reply to Alexey Proskuryakov from comment #2)
> > run-webkit-tests t241282 http/wpt/fetch/response-opaque-clone.html --iterations 2500 -f
> 
> Just to double check, does this not reproduce with r240157? I want to make
> sure that the repro case captures what regressed, not some other issue.

It appears the test was changed in 240158, so any test run prior to 240158 (i.e. 240157) is a 100% failure rate. So it appears the point of regression is 240158, when the test was modified.

[Shawn Roberts]

Some additional repro steps:

Did some additional testing with a single child process to make sure it's not a full parallel issue.

It is highly flaky on Mac, only 1 out of ~2500 iterations are a failure.

Reproducible locally with the most current revision r242721 and r240158 with :

run-webkit-tests http/wpt/fetch/response-opaque-clone.html --iterations 2500 --child-processes 1

Newer errors on the build logs show the same as my local diff :

@@ -1,3 +1,4 @@
+CONSOLE MESSAGE: Cache API operation failed: Failed writing data to the file system
 
-PASS Check opaque response can be cloned and cached correctly 
+FAIL Check opaque response can be cloned and cached correctly promise_test: Unhandled rejection with value: object "TypeError: Failed writing data to the file system"

[Shawn Roberts]

When running test locally on r240157 with:

run-webkit-tests http/wpt/fetch/response-opaque-clone.html --iterations 10 --child-processes 1

Getting 10/10 failures : failed unexpectedly (text diff)

Diff :

@@ -1,3 +1,3 @@
 
-PASS Check opaque response can be cloned and cached correctly 
+FAIL Check opaque response can be cloned and cached correctly assert_equals: fetched response url expected "" but got "http://127.0.0.1:8800/"

[Shawn Roberts]

Test started to become a flaky failure on iOS Simulator as well.

Reproduced with :

run-webkit-tests http/wpt/fetch/response-opaque-clone.html --iterations 500 --child-processes 1 --ios-simulator

Failing approximately 2/500 tries.

Same diff locally as seen on server:

+CONSOLE MESSAGE: Cache API operation failed: Failed writing data to the file system
 
-PASS Check opaque response can be cloned and cached correctly 
+FAIL Check opaque response can be cloned and cached correctly promise_test: Unhandled rejection with value: object "TypeError: Failed writing data to the file system"

When running on r240157 with:

run-webkit-tests http/wpt/fetch/response-opaque-clone.html --iterations 10 --child-processes 1 --ios-simulator

10 out 10 failures : failed unexpectedly (text diff)

Diff:

@@ -1,3 +1,3 @@
 
-PASS Check opaque response can be cloned and cached correctly 
+FAIL Check opaque response can be cloned and cached correctly assert_equals: fetched response url expected "" but got "http://127.0.0.1:8800/"

[youenn fablet]

Test is no longer failing on Mac.
It is sometimes testharness timing out on iOS simulator. Sometimes, it passes after running for 20 seconds which is a lot. Maybe it should marked as slow in ios simulator.

[youenn fablet]

Created attachment 367004 [details]
Patch

[EWS Watchlist]

Comment on attachment 367004 [details]
Patch

Attachment 367004 [details] did not pass mac-ews (mac):
Output: https://webkit-queues.webkit.org/results/11812927

New failing tests:
http/wpt/fetch/response-opaque-clone.html

[EWS Watchlist]

Created attachment 367008 [details]
Archive of layout-test-results from ews102 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews102  Port: mac-highsierra  Platform: Mac OS X 10.13.6

[EWS Watchlist]

Comment on attachment 367004 [details]
Patch

Attachment 367004 [details] did not pass mac-debug-ews (mac):
Output: https://webkit-queues.webkit.org/results/11812912

New failing tests:
http/wpt/fetch/response-opaque-clone.html

[EWS Watchlist]

Created attachment 367010 [details]
Archive of layout-test-results from ews117 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews117  Port: mac-highsierra  Platform: Mac OS X 10.13.6

[EWS Watchlist]

Comment on attachment 367004 [details]
Patch

Attachment 367004 [details] did not pass win-ews (win):
Output: https://webkit-queues.webkit.org/results/11813834

New failing tests:
http/wpt/fetch/response-opaque-clone.html

[EWS Watchlist]

Created attachment 367023 [details]
Archive of layout-test-results from ews200 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews200  Port: win-future  Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit

[youenn fablet]

Created attachment 367050 [details]
Patch

[Alex Christensen]

Comment on attachment 367050 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=367050&action=review

> LayoutTests/ChangeLog:11
> +        It sometimes passes after being runned for 20 seconds.

after running

[youenn fablet]

Created attachment 367152 [details]
Patch for landing

[WebKit Commit Bot]

The commit-queue encountered the following flaky tests while processing attachment 367152 [details]:

imported/w3c/web-platform-tests/xhr/event-upload-progress.htm bug 196736 (author: [e-mail])
The commit-queue is continuing to process your patch.

[WebKit Commit Bot]

Comment on attachment 367152 [details]
Patch for landing

Clearing flags on attachment: 367152

Committed r244150: <https://trac.webkit.org/changeset/244150>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.