The following 4 tests are flaky text failures on iOS: imported/w3c/web-platform-tests/fetch/api/policies/referrer-no-referrer-service-worker.https.html imported/w3c/web-platform-tests/fetch/api/request/destination/fetch-destination-frame.https.html imported/w3c/web-platform-tests/fetch/api/request/destination/fetch-destination-iframe.https.html imported/w3c/web-platform-tests/fetch/api/request/request-reset-attributes.https.html HISTORY: https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Ffetch%2Fapi%2Fpolicies%2Freferrer-no-referrer-service-worker.https.html&test=imported%2Fw3c%2Fweb-platform-tests%2Ffetch%2Fapi%2Frequest%2Fdestination%2Ffetch-destination-frame.https.html&test=imported%2Fw3c%2Fweb-platform-tests%2Ffetch%2Fapi%2Frequest%2Fdestination%2Ffetch-destination-iframe.https.html&test=imported%2Fw3c%2Fweb-platform-tests%2Ffetch%2Fapi%2Frequest%2Frequest-reset-attributes.https.html&platform=ios DIFF URLS: https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r286958%20(727)/imported/w3c/web-platform-tests/fetch/api/policies/referrer-no-referrer-service-worker.https-pretty-diff.html https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r286958%20(727)/imported/w3c/web-platform-tests/fetch/api/request/destination/fetch-destination-frame.https-pretty-diff.html https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r286958%20(727)/imported/w3c/web-platform-tests/fetch/api/request/destination/fetch-destination-iframe.https-pretty-diff.html https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r286958%20(727)/imported/w3c/web-platform-tests/fetch/api/request/request-reset-attributes.https-pretty-diff.html Each diff appears to have the following in common: +Harness Error (TIMEOUT), message = null I have attached all text diffs to this bug.
Created attachment 447079 [details] Text Diffs
They are all flakey, but they all appear to have either the same failure history, or at least very similar failure points when observing their history.
<rdar://problem/86438914>
After looking deeper into the history, it appears there are actually 4 more associated tests that also appear to have a very similar failure history (eg. the tests fail at the same time when they fail.) The additional 4 tests are: imported/w3c/web-platform-tests/fetch/api/policies/referrer-origin-service-worker.https.html imported/w3c/web-platform-tests/fetch/api/policies/referrer-origin-when-cross-origin-service-worker.https.html imported/w3c/web-platform-tests/fetch/api/policies/referrer-unsafe-url-service-worker.https.html imported/w3c/web-platform-tests/fetch/api/request/destination/fetch-destination-worker.https.html HISTORY: https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Ffetch%2Fapi%2Fpolicies%2Freferrer-origin-service-worker.https.html&test=imported%2Fw3c%2Fweb-platform-tests%2Ffetch%2Fapi%2Fpolicies%2Freferrer-origin-when-cross-origin-service-worker.https.html&test=imported%2Fw3c%2Fweb-platform-tests%2Ffetch%2Fapi%2Fpolicies%2Freferrer-unsafe-url-service-worker.https.html&test=imported%2Fw3c%2Fweb-platform-tests%2Ffetch%2Fapi%2Frequest%2Fdestination%2Ffetch-destination-worker.https.html&platform=ios DIFF URLS: https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r284862%20(318)/imported/w3c/web-platform-tests/fetch/api/policies/referrer-no-referrer-service-worker.https-pretty-diff.html https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r284862%20(318)/imported/w3c/web-platform-tests/fetch/api/request/destination/fetch-destination-frame.https-pretty-diff.html https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r284862%20(318)/imported/w3c/web-platform-tests/fetch/api/request/destination/fetch-destination-iframe.https-pretty-diff.html https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r284862%20(318)/imported/w3c/web-platform-tests/fetch/api/request/request-reset-attributes.https-pretty-diff.html These four additional tests all have the same thing in common in their diffs that the original 4 did posted above: +Harness Error (TIMEOUT), message = null I have attached all text diffs to this bug.
All of these tests had their first reported failure at r284862, and have continued since. I don't believe r284862 is what caused them to start failing, that's just what their history highlights. That revision was close to when we started testing iOS 15 on Open Source, so it appears that these are all just flaky tests on iOS 15.
Created attachment 447087 [details] 8 Text Diffs for failing tests
I was not able to reproduce any of these failures at Production or Debug iOS 15 ToT. I even took it up to 1000 iterations for each test, and they all passed. These tests randomly fail, and do not appear to have a specific regression point. I'm grouping them together because they have either the same failure history, or extremely similar failure history. They all also appear to deal with or be service worker tests. I have marked the expectations for these tests as Pass, Failure here: https://trac.webkit.org/changeset/286997/webkit
All these tests are service worker tests that seem to fail registering/unregistering service workers on simulator.
imported/w3c/web-platform-tests/fetch/api/policies/referrer-no-referrer-service-worker.https.html, imported/w3c/web-platform-tests/fetch/api/request/request-reset-attributes.https.html and imported/w3c/web-platform-tests/fetch/api/request/destination/fetch-destination-iframe.https.html do not seem flaky anymore. imported/w3c/web-platform-tests/fetch/api/request/destination/fetch-destination-frame.https.html seems to crash sometimes (GPUProcess) but without any actual crash log.
imported/w3c/web-platform-tests/fetch/api/policies/referrer-origin-service-worker.https.html imported/w3c/web-platform-tests/fetch/api/policies/referrer-origin-when-cross-origin-service-worker.https.html imported/w3c/web-platform-tests/fetch/api/policies/referrer-unsafe-url-service-worker.https.html imported/w3c/web-platform-tests/fetch/api/request/destination/fetch-destination-worker.https.html All these tests are passing except imported/w3c/web-platform-tests/fetch/api/request/destination/fetch-destination-worker.https.html which sometimes crash. I'll fix the crash.
Fixing the crash at https://bugs.webkit.org/show_bug.cgi?id=241776. The test expectations should probably be unflaked.
Pull request: https://github.com/WebKit/WebKit/pull/1634
Committed r295673 (251678@main): <https://commits.webkit.org/251678@main> Reviewed commits have been landed. Closing PR #1634 and removing active labels.