Bug 240074

Summary: REGRESSION (r293506): [ iOS ][ macOS ] imported/w3c/web-platform-tests/service-workers/service-worker/registration-updateviacache.https.html is a flaky failure
Product: WebKit Reporter: Karl Rackler <rackler>
Component: Service WorkersAssignee: youenn fablet <youennf>
Status: RESOLVED FIXED    
Severity: Normal CC: cdumez, webkit-bot-watchers-bugzilla, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=180982
https://bugs.webkit.org/show_bug.cgi?id=179138
https://github.com/web-platform-tests/wpt/pull/34506
https://bugs.webkit.org/show_bug.cgi?id=243410

Description Karl Rackler 2022-05-04 10:53:08 PDT
Description:
imported/w3c/web-platform-tests/service-workers/service-worker/registration-updateviacache.https.html

The first failure that I saw on the dashboard for iOS was 4/27/2022 at r293507, but this does not appear relevant to causing this issue.  The first failure that I saw on the dashboard for macOS was 4/27/2022 at r293506, but this does not appear relevant to causing this issue either.   r293506 at https://commits.webkit.org/250037@main modified ServiceWorker, which probably introduced the flakiness/consistent failure. 

History:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fservice-workers%2Fservice-worker%2Fregistration-updateviacache.https.html&platform=ios&platform=mac&limit=50000

Diff:
--- /Volumes/Data/worker/Apple-iOS-15-Simulator-Release-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/service-workers/service-worker/registration-updateviacache.https-expected.txt
+++ /Volumes/Data/worker/Apple-iOS-15-Simulator-Release-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/service-workers/service-worker/registration-updateviacache.https-actual.txt
@@ -1,3 +1,4 @@
+
 
 PASS register-with-updateViaCache-undefined
 PASS register-with-updateViaCache-imports
@@ -5,11 +6,11 @@
 PASS register-with-updateViaCache-none
 PASS register-with-updateViaCache-undefined-then-undefined
 PASS register-with-updateViaCache-undefined-then-imports
-PASS register-with-updateViaCache-undefined-then-all
+FAIL register-with-updateViaCache-undefined-then-all assert_equals: No new service worker expected null but got object "[object ServiceWorker]"
 PASS register-with-updateViaCache-undefined-then-none
 PASS register-with-updateViaCache-imports-then-undefined
 PASS register-with-updateViaCache-imports-then-imports
-PASS register-with-updateViaCache-imports-then-all
+FAIL register-with-updateViaCache-imports-then-all assert_equals: No new service worker expected null but got object "[object ServiceWorker]"
 PASS register-with-updateViaCache-imports-then-none
 PASS register-with-updateViaCache-all-then-undefined
 PASS register-with-updateViaCache-all-then-imports
@@ -17,7 +18,7 @@
 PASS register-with-updateViaCache-all-then-none
 PASS register-with-updateViaCache-none-then-undefined
 PASS register-with-updateViaCache-none-then-imports
-PASS register-with-updateViaCache-none-then-all
+FAIL register-with-updateViaCache-none-then-all assert_equals: No new service worker expected null but got object "[object ServiceWorker]"
 PASS register-with-updateViaCache-none-then-none
 PASS access-updateViaCache-after-unregister-undefined
 PASS access-updateViaCache-after-unregister-imports
Comment 1 Radar WebKit Bug Importer 2022-05-04 10:54:05 PDT
<rdar://problem/92742526>
Comment 2 Karl Rackler 2022-05-04 11:06:16 PDT
I have marked this test as a flaky failure while this issue is investigated.
Comment 3 Karl Rackler 2022-05-04 11:11:57 PDT
Pull request: https://github.com/WebKit/WebKit/pull/497
Comment 4 Karl Rackler 2022-05-04 11:15:18 PDT
(In reply to Karl Rackler from comment #0)
> Description:
> imported/w3c/web-platform-tests/service-workers/service-worker/registration-
> updateviacache.https.html
> 
> The first failure that I saw on the dashboard for iOS was 4/27/2022 at
> r293507, but this does not appear relevant to causing this issue.  The first
> failure that I saw on the dashboard for macOS was 4/27/2022 at r293506, but
> this does not appear relevant to causing this issue either.   r293506 at
> https://commits.webkit.org/250037@main modified ServiceWorker, which
> probably introduced the flakiness/consistent failure. 
> 
> History:
> https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-
> platform-tests%2Fservice-workers%2Fservice-worker%2Fregistration-
> updateviacache.https.html&platform=ios&platform=mac&limit=50000
> 
> Diff:
> ---
> /Volumes/Data/worker/Apple-iOS-15-Simulator-Release-WK2-Tests/build/layout-
> test-results/imported/w3c/web-platform-tests/service-workers/service-worker/
> registration-updateviacache.https-expected.txt
> +++
> /Volumes/Data/worker/Apple-iOS-15-Simulator-Release-WK2-Tests/build/layout-
> test-results/imported/w3c/web-platform-tests/service-workers/service-worker/
> registration-updateviacache.https-actual.txt
> @@ -1,3 +1,4 @@
> +
>  
>  PASS register-with-updateViaCache-undefined
>  PASS register-with-updateViaCache-imports
> @@ -5,11 +6,11 @@
>  PASS register-with-updateViaCache-none
>  PASS register-with-updateViaCache-undefined-then-undefined
>  PASS register-with-updateViaCache-undefined-then-imports
> -PASS register-with-updateViaCache-undefined-then-all
> +FAIL register-with-updateViaCache-undefined-then-all assert_equals: No new
> service worker expected null but got object "[object ServiceWorker]"
>  PASS register-with-updateViaCache-undefined-then-none
>  PASS register-with-updateViaCache-imports-then-undefined
>  PASS register-with-updateViaCache-imports-then-imports
> -PASS register-with-updateViaCache-imports-then-all
> +FAIL register-with-updateViaCache-imports-then-all assert_equals: No new
> service worker expected null but got object "[object ServiceWorker]"
>  PASS register-with-updateViaCache-imports-then-none
>  PASS register-with-updateViaCache-all-then-undefined
>  PASS register-with-updateViaCache-all-then-imports
> @@ -17,7 +18,7 @@
>  PASS register-with-updateViaCache-all-then-none
>  PASS register-with-updateViaCache-none-then-undefined
>  PASS register-with-updateViaCache-none-then-imports
> -PASS register-with-updateViaCache-none-then-all
> +FAIL register-with-updateViaCache-none-then-all assert_equals: No new
> service worker expected null but got object "[object ServiceWorker]"
>  PASS register-with-updateViaCache-none-then-none
>  PASS access-updateViaCache-after-unregister-undefined
>  PASS access-updateViaCache-after-unregister-imports

Correct Description to: 
The first failure that I saw on the dashboard for iOS was 4/27/2022 at r293507, but this does not appear relevant to causing this issue.  The first failure that I saw on the dashboard for macOS was 4/27/2022 at r293506, r293506 at https://commits.webkit.org/250037@main modified ServiceWorker, which probably introduced the flakiness failure.
Comment 5 Karl Rackler 2022-05-05 08:14:45 PDT
Pull request: https://github.com/WebKit/WebKit/pull/523
Comment 6 EWS 2022-05-05 08:19:14 PDT
Test gardening commit r293834 (250307@main): <https://commits.webkit.org/250307@main>

Reviewed commits have been landed. Closing PR #523 and removing active labels.
Comment 8 youenn fablet 2022-06-21 03:43:53 PDT
Pull request: https://github.com/WebKit/WebKit/pull/1646
Comment 9 youenn fablet 2022-06-21 06:19:48 PDT
Submitted web-platform-tests pull request: https://github.com/web-platform-tests/wpt/pull/34506
Comment 10 EWS 2022-06-22 10:58:12 PDT
Committed r295737 (251742@main): <https://commits.webkit.org/251742@main>

Reviewed commits have been landed. Closing PR #1646 and removing active labels.
Comment 11 Chris Dumez 2022-08-04 12:18:20 PDT
According to the flakiness dashboard, this test is still flaky. Also, it is still marked as flaky in TestExpections.
Comment 12 Chris Dumez 2022-08-04 12:19:44 PDT
This in an example of recent failure:

--- /Volumes/Data/worker/Apple-iPadOS-15-Simulator-Release-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/service-workers/service-worker/registration-updateviacache.https-expected.txt
+++ /Volumes/Data/worker/Apple-iPadOS-15-Simulator-Release-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/service-workers/service-worker/registration-updateviacache.https-actual.txt
@@ -1,3 +1,4 @@
+
 
 PASS register-with-updateViaCache-undefined
 PASS register-with-updateViaCache-imports
@@ -16,7 +17,7 @@
 PASS register-with-updateViaCache-all-then-all
 PASS register-with-updateViaCache-all-then-none
 PASS register-with-updateViaCache-none-then-undefined
-PASS register-with-updateViaCache-none-then-imports
+FAIL register-with-updateViaCache-none-then-imports assert_equals: Imported script should be the same expected 1659576339.230555 but got 1659576336.951165
 PASS register-with-updateViaCache-none-then-all
 PASS register-with-updateViaCache-none-then-none
 PASS access-updateViaCache-after-unregister-undefined