Bug 240074 – REGRESSION (r293506): [ iOS ][ macOS ] imported/w3c/web-platform-tests/service-workers/service-worker/registration-updateviacache.https.html is a flaky failure

• Status: RESOLVED
• Resolution: FIXED
• Component: Service Workers

[Karl Rackler]

Description:
imported/w3c/web-platform-tests/service-workers/service-worker/registration-updateviacache.https.html

The first failure that I saw on the dashboard for iOS was 4/27/2022 at r293507, but this does not appear relevant to causing this issue.  The first failure that I saw on the dashboard for macOS was 4/27/2022 at r293506, but this does not appear relevant to causing this issue either.   r293506 at https://commits.webkit.org/250037@main modified ServiceWorker, which probably introduced the flakiness/consistent failure. 

History:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fservice-workers%2Fservice-worker%2Fregistration-updateviacache.https.html&platform=ios&platform=mac&limit=50000

Diff:
--- /Volumes/Data/worker/Apple-iOS-15-Simulator-Release-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/service-workers/service-worker/registration-updateviacache.https-expected.txt
+++ /Volumes/Data/worker/Apple-iOS-15-Simulator-Release-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/service-workers/service-worker/registration-updateviacache.https-actual.txt
@@ -1,3 +1,4 @@
+
 
 PASS register-with-updateViaCache-undefined
 PASS register-with-updateViaCache-imports
@@ -5,11 +6,11 @@
 PASS register-with-updateViaCache-none
 PASS register-with-updateViaCache-undefined-then-undefined
 PASS register-with-updateViaCache-undefined-then-imports
-PASS register-with-updateViaCache-undefined-then-all
+FAIL register-with-updateViaCache-undefined-then-all assert_equals: No new service worker expected null but got object "[object ServiceWorker]"
 PASS register-with-updateViaCache-undefined-then-none
 PASS register-with-updateViaCache-imports-then-undefined
 PASS register-with-updateViaCache-imports-then-imports
-PASS register-with-updateViaCache-imports-then-all
+FAIL register-with-updateViaCache-imports-then-all assert_equals: No new service worker expected null but got object "[object ServiceWorker]"
 PASS register-with-updateViaCache-imports-then-none
 PASS register-with-updateViaCache-all-then-undefined
 PASS register-with-updateViaCache-all-then-imports
@@ -17,7 +18,7 @@
 PASS register-with-updateViaCache-all-then-none
 PASS register-with-updateViaCache-none-then-undefined
 PASS register-with-updateViaCache-none-then-imports
-PASS register-with-updateViaCache-none-then-all
+FAIL register-with-updateViaCache-none-then-all assert_equals: No new service worker expected null but got object "[object ServiceWorker]"
 PASS register-with-updateViaCache-none-then-none
 PASS access-updateViaCache-after-unregister-undefined
 PASS access-updateViaCache-after-unregister-imports

[Radar WebKit Bug Importer]

<rdar://problem/92742526>

[Karl Rackler]

I have marked this test as a flaky failure while this issue is investigated.

[Karl Rackler]

Pull request: https://github.com/WebKit/WebKit/pull/497

[Karl Rackler]

(In reply to Karl Rackler from comment #0)
> Description:
> imported/w3c/web-platform-tests/service-workers/service-worker/registration-
> updateviacache.https.html
> 
> The first failure that I saw on the dashboard for iOS was 4/27/2022 at
> r293507, but this does not appear relevant to causing this issue.  The first
> failure that I saw on the dashboard for macOS was 4/27/2022 at r293506, but
> this does not appear relevant to causing this issue either.   r293506 at
> https://commits.webkit.org/250037@main modified ServiceWorker, which
> probably introduced the flakiness/consistent failure. 
> 
> History:
> https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-
> platform-tests%2Fservice-workers%2Fservice-worker%2Fregistration-
> updateviacache.https.html&platform=ios&platform=mac&limit=50000
> 
> Diff:
> ---
> /Volumes/Data/worker/Apple-iOS-15-Simulator-Release-WK2-Tests/build/layout-
> test-results/imported/w3c/web-platform-tests/service-workers/service-worker/
> registration-updateviacache.https-expected.txt
> +++
> /Volumes/Data/worker/Apple-iOS-15-Simulator-Release-WK2-Tests/build/layout-
> test-results/imported/w3c/web-platform-tests/service-workers/service-worker/
> registration-updateviacache.https-actual.txt
> @@ -1,3 +1,4 @@
> +
>  
>  PASS register-with-updateViaCache-undefined
>  PASS register-with-updateViaCache-imports
> @@ -5,11 +6,11 @@
>  PASS register-with-updateViaCache-none
>  PASS register-with-updateViaCache-undefined-then-undefined
>  PASS register-with-updateViaCache-undefined-then-imports
> -PASS register-with-updateViaCache-undefined-then-all
> +FAIL register-with-updateViaCache-undefined-then-all assert_equals: No new
> service worker expected null but got object "[object ServiceWorker]"
>  PASS register-with-updateViaCache-undefined-then-none
>  PASS register-with-updateViaCache-imports-then-undefined
>  PASS register-with-updateViaCache-imports-then-imports
> -PASS register-with-updateViaCache-imports-then-all
> +FAIL register-with-updateViaCache-imports-then-all assert_equals: No new
> service worker expected null but got object "[object ServiceWorker]"
>  PASS register-with-updateViaCache-imports-then-none
>  PASS register-with-updateViaCache-all-then-undefined
>  PASS register-with-updateViaCache-all-then-imports
> @@ -17,7 +18,7 @@
>  PASS register-with-updateViaCache-all-then-none
>  PASS register-with-updateViaCache-none-then-undefined
>  PASS register-with-updateViaCache-none-then-imports
> -PASS register-with-updateViaCache-none-then-all
> +FAIL register-with-updateViaCache-none-then-all assert_equals: No new
> service worker expected null but got object "[object ServiceWorker]"
>  PASS register-with-updateViaCache-none-then-none
>  PASS access-updateViaCache-after-unregister-undefined
>  PASS access-updateViaCache-after-unregister-imports

Correct Description to: 
The first failure that I saw on the dashboard for iOS was 4/27/2022 at r293507, but this does not appear relevant to causing this issue.  The first failure that I saw on the dashboard for macOS was 4/27/2022 at r293506, r293506 at https://commits.webkit.org/250037@main modified ServiceWorker, which probably introduced the flakiness failure.

[Karl Rackler]

Pull request: https://github.com/WebKit/WebKit/pull/523

[EWS]

Test gardening commit r293834 (250307@main): <https://commits.webkit.org/250307@main>

Reviewed commits have been landed. Closing PR #523 and removing active labels.

[Sam Sneddon [:gsnedders]]

This is also consistently showing up as a failure on wpt.fyi as of STP146:

https://wpt.fyi/results/service-workers/service-worker/registration-updateviacache.https.html?q=%20seq%28%28status%3Apass%7Cstatus%3Aok%29%20%28status%3Apass%7Cstatus%3Aok%29%20%28status%3Apass%7Cstatus%3Aok%29%20%28status%3Apass%7Cstatus%3Aok%29%20%28status%3Apass%7Cstatus%3Aok%29%20%28status%3A%21pass%26status%3A%21ok%26status%3A%21unknown%29%20%28status%3A%21pass%26status%3A%21ok%26status%3A%21unknown%29%20%28status%3A%21pass%26status%3A%21ok%26status%3A%21unknown%29%20%28status%3A%21pass%26status%3A%21ok%26status%3A%21unknown%29%20%28status%3A%21pass%26status%3A%21ok%26status%3A%21unknown%29%29&run_id=5118614477733888&run_id=5690424244043776&run_id=5698072775491584&run_id=5702679396352000&run_id=5164587304353792&run_id=5682637401227264&run_id=5757110456418304&run_id=5713130930831360&run_id=5759361623261184&run_id=5743658283302912

[youenn fablet]

Pull request: https://github.com/WebKit/WebKit/pull/1646

[youenn fablet]

Submitted web-platform-tests pull request: https://github.com/web-platform-tests/wpt/pull/34506

[EWS]

Committed r295737 (251742@main): <https://commits.webkit.org/251742@main>

Reviewed commits have been landed. Closing PR #1646 and removing active labels.

[Chris Dumez]

According to the flakiness dashboard, this test is still flaky. Also, it is still marked as flaky in TestExpections.

[Chris Dumez]

This in an example of recent failure:

--- /Volumes/Data/worker/Apple-iPadOS-15-Simulator-Release-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/service-workers/service-worker/registration-updateviacache.https-expected.txt
+++ /Volumes/Data/worker/Apple-iPadOS-15-Simulator-Release-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/service-workers/service-worker/registration-updateviacache.https-actual.txt
@@ -1,3 +1,4 @@
+
 
 PASS register-with-updateViaCache-undefined
 PASS register-with-updateViaCache-imports
@@ -16,7 +17,7 @@
 PASS register-with-updateViaCache-all-then-all
 PASS register-with-updateViaCache-all-then-none
 PASS register-with-updateViaCache-none-then-undefined
-PASS register-with-updateViaCache-none-then-imports
+FAIL register-with-updateViaCache-none-then-imports assert_equals: Imported script should be the same expected 1659576339.230555 but got 1659576336.951165
 PASS register-with-updateViaCache-none-then-all
 PASS register-with-updateViaCache-none-then-none
 PASS access-updateViaCache-after-unregister-undefined