Bug 228327

Summary: [iOS Release] 2 imported/w3c/web-platform-tests/worklets tests are flaky failing
Product: WebKit Reporter: ayumi_kojima
Component: New BugsAssignee: Nobody <webkit-unassigned>
Status: NEW ---    
Severity: Normal CC: cdumez, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: iPhone / iPad   
OS: Unspecified   

Description ayumi_kojima 2021-07-27 09:03:47 PDT
imported/w3c/web-platform-tests/worklets/audio-worklet-service-worker-interception.https.html

Is flaky failing on iOS 14 E Simulator wk2 Release.

History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fworklets%2Faudio-worklet-service-worker-interception.https.html&platform=ios

Diff: 

--- /Volumes/Data/worker/ipados-simulator-14-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/worklets/audio-worklet-service-worker-interception.https-expected.txt
+++ /Volumes/Data/worker/ipados-simulator-14-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/worklets/audio-worklet-service-worker-interception.https-actual.txt
@@ -1,5 +1,7 @@
 
-PASS addModule() on a controlled document should be intercepted by a service worker.
-PASS addModule() on a non-controlled document should not be intercepted by a service worker even if the script is under the service worker scope.
-PASS Static import should be intercepted by a service worker.
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT addModule() on a controlled document should be intercepted by a service worker. Test timed out
+NOTRUN addModule() on a non-controlled document should not be intercepted by a service worker even if the script is under the service worker scope.
+NOTRUN Static import should be intercepted by a service worker.
+
Comment 1 ayumi_kojima 2021-07-27 09:32:53 PDT
I was not able to reproduce the failure on iOS 15 simulator on my local machine using: 

run-webkit-tests --ios-simulator --release --iterations 500  --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 imported/w3c/web-platform-tests/worklets/audio-worklet-service-worker-interception.https.html

The test timed out instead of failing.

It seems it's been flaky as long as I can see in the history (the oldest revision is r277101). The test is imported at r270016.

Updated test expectations https://trac.webkit.org/changeset/280340/webkit
Comment 2 Radar WebKit Bug Importer 2021-07-27 09:35:54 PDT
<rdar://problem/81167040>
Comment 3 ayumi_kojima 2021-07-27 09:45:56 PDT
imported/w3c/web-platform-tests/worklets/paint-worklet-service-worker-interception.https.html

Is also flaky failing iOS 14 E Simulator wk2 Release. 

History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fworklets%2Fpaint-worklet-service-worker-interception.https.html&platform=ios

Diff: 

--- /Volumes/Data/worker/ipados-simulator-14-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/worklets/paint-worklet-service-worker-interception.https-expected.txt
+++ /Volumes/Data/worker/ipados-simulator-14-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/worklets/paint-worklet-service-worker-interception.https-actual.txt
@@ -1,8 +1,7 @@
-CONSOLE MESSAGE: Error: Script error.
-CONSOLE MESSAGE: Error: Script error.
-CONSOLE MESSAGE: Error: Script error.
 
-PASS addModule() on a controlled document should be intercepted by a service worker.
-FAIL addModule() on a non-controlled document should not be intercepted by a service worker even if the script is under the service worker scope. assert_equals: expected "REJECTED" but got "RESOLVED"
-PASS Static import should be intercepted by a service worker.
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT addModule() on a controlled document should be intercepted by a service worker. Test timed out
+NOTRUN addModule() on a non-controlled document should not be intercepted by a service worker even if the script is under the service worker scope.
+NOTRUN Static import should be intercepted by a service worker.
+
Comment 4 ayumi_kojima 2021-07-27 10:26:15 PDT
Updated test expectations https://trac.webkit.org/changeset/280343/webkit