Bug 201190 – [iOS] Layout Test imported/w3c/web-platform-tests/service-workers/service-worker/registration-mime-types.https.html is flaky

• Status: RESOLVED
• Resolution: FIXED
• Component: Service Workers

[Ryan Haddad]

The following layout test is flaky on iOS Simulator

imported/w3c/web-platform-tests/service-workers/service-worker/registration-mime-types.https.html

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fservice-workers%2Fservice-worker%2Fregistration-mime-types.https.html

--- /Volumes/Data/slave/ios-simulator-12-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/service-workers/service-worker/registration-mime-types.https-expected.txt
+++ /Volumes/Data/slave/ios-simulator-12-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/service-workers/service-worker/registration-mime-types.https-actual.txt
@@ -1,38 +1,5 @@
-
-PASS Registering script with no MIME type 
-PASS Registering script with bad MIME type 
-PASS Registering script that imports script with no MIME type 
-PASS Registering script that imports script with bad MIME type 
-PASS Registering script with good MIME type application/ecmascript 
-PASS Registering script that imports script with good MIME type application/ecmascript 
-PASS Registering script with good MIME type application/javascript 
-PASS Registering script that imports script with good MIME type application/javascript 
-PASS Registering script with good MIME type application/x-ecmascript 
-PASS Registering script that imports script with good MIME type application/x-ecmascript 
-PASS Registering script with good MIME type application/x-javascript 
-PASS Registering script that imports script with good MIME type application/x-javascript 
-PASS Registering script with good MIME type text/ecmascript 
-PASS Registering script that imports script with good MIME type text/ecmascript 
-PASS Registering script with good MIME type text/javascript 
-PASS Registering script that imports script with good MIME type text/javascript 
-PASS Registering script with good MIME type text/javascript1.0 
-PASS Registering script that imports script with good MIME type text/javascript1.0 
-PASS Registering script with good MIME type text/javascript1.1 
-PASS Registering script that imports script with good MIME type text/javascript1.1 
-PASS Registering script with good MIME type text/javascript1.2 
-PASS Registering script that imports script with good MIME type text/javascript1.2 
-PASS Registering script with good MIME type text/javascript1.3 
-PASS Registering script that imports script with good MIME type text/javascript1.3 
-PASS Registering script with good MIME type text/javascript1.4 
-PASS Registering script that imports script with good MIME type text/javascript1.4 
-PASS Registering script with good MIME type text/javascript1.5 
-PASS Registering script that imports script with good MIME type text/javascript1.5 
-PASS Registering script with good MIME type text/jscript 
-PASS Registering script that imports script with good MIME type text/jscript 
-PASS Registering script with good MIME type text/livescript 
-PASS Registering script that imports script with good MIME type text/livescript 
-PASS Registering script with good MIME type text/x-ecmascript 
-PASS Registering script that imports script with good MIME type text/x-ecmascript 
-PASS Registering script with good MIME type text/x-javascript 
-PASS Registering script that imports script with good MIME type text/x-javascript 
-
+layer at (0,0) size 800x600
+  RenderView at (0,0) size 800x600
+layer at (0,0) size 800x600
+  RenderBlock {HTML} at (0,0) size 800x600
+    RenderBody {BODY} at (8,8) size 784x584

[Ryan Haddad]

This test appears to have been flaky for quite a while.

[Radar WebKit Bug Importer]

<rdar://problem/54755541>

[Radar WebKit Bug Importer]

<rdar://problem/54755542>

[Alexey Proskuryakov]

stderr says:

2019-08-29 15:45:14.407 com.apple.WebKit.Networking.Development[9397:54056641] NSURLSession/NSURLConnection HTTP load failed (kCFStreamErrorDomainSSL, -9858)

This is consistent with the diff, which looks like failing to load test harness subresource.

[Ryan Haddad]

Marked test as flaky in https://trac.webkit.org/changeset/249744/webkit

[Chris Dumez]

(In reply to Alexey Proskuryakov from comment #4)
> stderr says:
> 
> 2019-08-29 15:45:14.407
> com.apple.WebKit.Networking.Development[9397:54056641]
> NSURLSession/NSURLConnection HTTP load failed (kCFStreamErrorDomainSSL,
> -9858)
> 
> This is consistent with the diff, which looks like failing to load test
> harness subresource.

@Alex & Youenn: Didn't we see this kind of SSL errors shortly after we started WPT tests? Do you remember what was caused this and how we made this go away?

[Chris Dumez]

It seems it registers 2 service workers successfully and fails on the third one:
Registering script that imports script with no MIME type

[Chris Dumez]

Could not reproduce locally with:
run-webkit-tests --ios-simulator --debug --no-build --repeat-each=100 imported/w3c/web-platform-tests/service-workers/service-worker/registration-mime-types.https.html -g

[Ryan Haddad]

The test is consistently passing now. Removed expectation in r259782.