Bug 194525 - Layout Test http/wpt/fetch/response-opaque-clone.html is sometimes timing out on iOS simulator
Summary: Layout Test http/wpt/fetch/response-opaque-clone.html is sometimes timing out...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: youenn fablet
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2019-02-11 17:17 PST by Shawn Roberts
Modified: 2019-04-10 14:16 PDT (History)
10 users (show)

See Also:


Attachments
Patch (5.15 KB, patch)
2019-04-08 17:39 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews102 for mac-highsierra (2.43 MB, application/zip)
2019-04-08 18:42 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews117 for mac-highsierra (2.38 MB, application/zip)
2019-04-08 19:09 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews200 for win-future (12.84 MB, application/zip)
2019-04-08 20:30 PDT, Build Bot
no flags Details
Patch (5.31 KB, patch)
2019-04-09 10:18 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Patch for landing (5.31 KB, patch)
2019-04-10 13:19 PDT, youenn fablet
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Shawn Roberts 2019-02-11 17:17:16 PST
The following layout test is flaky on Mac WK2

http/wpt/fetch/response-opaque-clone.html

Probable cause:

Test appears to have been changed with revision 240158. Is a flaky failure with several versions tested from 240158 to 241282

Reproduced with:

run-webkit-tests t241282 http/wpt/fetch/response-opaque-clone.html --iterations 2500 -f

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Fwpt%2Ffetch%2Fresponse-opaque-clone.html

Diff:

--- /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/http/wpt/fetch/response-opaque-clone-expected.txt
+++ /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/http/wpt/fetch/response-opaque-clone-actual.txt
@@ -1,3 +1,5 @@
+#PID UNRESPONSIVE - com.apple.WebKit.WebContent.Development (pid 38099)
+FAIL: Timed out waiting for notifyDone to be called
 
-PASS Check opaque response can be cloned and cached correctly 
-
+#EOF
+#EOF
Comment 1 Radar WebKit Bug Importer 2019-02-12 13:17:44 PST
<rdar://problem/48012229>
Comment 2 Alexey Proskuryakov 2019-02-15 23:24:00 PST
> run-webkit-tests t241282 http/wpt/fetch/response-opaque-clone.html --iterations 2500 -f

Just to double check, does this not reproduce with r240157? I want to make sure that the repro case captures what regressed, not some other issue.
Comment 3 Shawn Roberts 2019-02-17 16:40:58 PST
(In reply to Alexey Proskuryakov from comment #2)
> > run-webkit-tests t241282 http/wpt/fetch/response-opaque-clone.html --iterations 2500 -f
> 
> Just to double check, does this not reproduce with r240157? I want to make
> sure that the repro case captures what regressed, not some other issue.

It appears the test was changed in 240158, so any test run prior to 240158 (i.e. 240157) is a 100% failure rate. So it appears the point of regression is 240158, when the test was modified.
Comment 4 Shawn Roberts 2019-03-11 14:21:45 PDT
Some additional repro steps:

Did some additional testing with a single child process to make sure it's not a full parallel issue.

It is highly flaky on Mac, only 1 out of ~2500 iterations are a failure.

Reproducible locally with the most current revision r242721 and r240158 with :

run-webkit-tests http/wpt/fetch/response-opaque-clone.html --iterations 2500 --child-processes 1

Newer errors on the build logs show the same as my local diff :

@@ -1,3 +1,4 @@
+CONSOLE MESSAGE: Cache API operation failed: Failed writing data to the file system
 
-PASS Check opaque response can be cloned and cached correctly 
+FAIL Check opaque response can be cloned and cached correctly promise_test: Unhandled rejection with value: object "TypeError: Failed writing data to the file system"
Comment 5 Shawn Roberts 2019-03-11 14:23:33 PDT
When running test locally on r240157 with:

run-webkit-tests http/wpt/fetch/response-opaque-clone.html --iterations 10 --child-processes 1

Getting 10/10 failures : failed unexpectedly (text diff)

Diff :

@@ -1,3 +1,3 @@
 
-PASS Check opaque response can be cloned and cached correctly 
+FAIL Check opaque response can be cloned and cached correctly assert_equals: fetched response url expected "" but got "http://127.0.0.1:8800/"
Comment 6 Shawn Roberts 2019-03-11 14:39:10 PDT
Test started to become a flaky failure on iOS Simulator as well.

Reproduced with :

run-webkit-tests http/wpt/fetch/response-opaque-clone.html --iterations 500 --child-processes 1 --ios-simulator

Failing approximately 2/500 tries.

Same diff locally as seen on server:

+CONSOLE MESSAGE: Cache API operation failed: Failed writing data to the file system
 
-PASS Check opaque response can be cloned and cached correctly 
+FAIL Check opaque response can be cloned and cached correctly promise_test: Unhandled rejection with value: object "TypeError: Failed writing data to the file system"

When running on r240157 with:

run-webkit-tests http/wpt/fetch/response-opaque-clone.html --iterations 10 --child-processes 1 --ios-simulator

10 out 10 failures : failed unexpectedly (text diff)

Diff:

@@ -1,3 +1,3 @@
 
-PASS Check opaque response can be cloned and cached correctly 
+FAIL Check opaque response can be cloned and cached correctly assert_equals: fetched response url expected "" but got "http://127.0.0.1:8800/"
Comment 7 youenn fablet 2019-04-08 17:30:25 PDT
Test is no longer failing on Mac.
It is sometimes testharness timing out on iOS simulator. Sometimes, it passes after running for 20 seconds which is a lot. Maybe it should marked as slow in ios simulator.
Comment 8 youenn fablet 2019-04-08 17:39:29 PDT
Created attachment 367004 [details]
Patch
Comment 9 Build Bot 2019-04-08 18:42:21 PDT
Comment on attachment 367004 [details]
Patch

Attachment 367004 [details] did not pass mac-ews (mac):
Output: https://webkit-queues.webkit.org/results/11812927

New failing tests:
http/wpt/fetch/response-opaque-clone.html
Comment 10 Build Bot 2019-04-08 18:42:22 PDT
Created attachment 367008 [details]
Archive of layout-test-results from ews102 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews102  Port: mac-highsierra  Platform: Mac OS X 10.13.6
Comment 11 Build Bot 2019-04-08 19:09:43 PDT
Comment on attachment 367004 [details]
Patch

Attachment 367004 [details] did not pass mac-debug-ews (mac):
Output: https://webkit-queues.webkit.org/results/11812912

New failing tests:
http/wpt/fetch/response-opaque-clone.html
Comment 12 Build Bot 2019-04-08 19:09:45 PDT
Created attachment 367010 [details]
Archive of layout-test-results from ews117 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews117  Port: mac-highsierra  Platform: Mac OS X 10.13.6
Comment 13 Build Bot 2019-04-08 20:30:34 PDT
Comment on attachment 367004 [details]
Patch

Attachment 367004 [details] did not pass win-ews (win):
Output: https://webkit-queues.webkit.org/results/11813834

New failing tests:
http/wpt/fetch/response-opaque-clone.html
Comment 14 Build Bot 2019-04-08 20:30:45 PDT
Created attachment 367023 [details]
Archive of layout-test-results from ews200 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews200  Port: win-future  Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit
Comment 15 youenn fablet 2019-04-09 10:18:33 PDT
Created attachment 367050 [details]
Patch
Comment 16 Alex Christensen 2019-04-10 12:47:59 PDT
Comment on attachment 367050 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=367050&action=review

> LayoutTests/ChangeLog:11
> +        It sometimes passes after being runned for 20 seconds.

after running
Comment 17 youenn fablet 2019-04-10 13:19:43 PDT
Created attachment 367152 [details]
Patch for landing
Comment 18 WebKit Commit Bot 2019-04-10 14:16:00 PDT
The commit-queue encountered the following flaky tests while processing attachment 367152 [details]:

imported/w3c/web-platform-tests/xhr/event-upload-progress.htm bug 196736 (author: youennf@gmail.com)
The commit-queue is continuing to process your patch.
Comment 19 WebKit Commit Bot 2019-04-10 14:16:48 PDT
Comment on attachment 367152 [details]
Patch for landing

Clearing flags on attachment: 367152

Committed r244150: <https://trac.webkit.org/changeset/244150>
Comment 20 WebKit Commit Bot 2019-04-10 14:16:50 PDT
All reviewed patches have been landed.  Closing bug.