Bug 209281 – [ iOS wk2 and Mac wk2 ] imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html is flaky failing.

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Jason Lawrence]

imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html

Description:
This test is flaky failing on Mac wk2 and iOS wk2. The flaky failures are visible since the introduction of the test.

History:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Ffetch%2Fstale-while-revalidate%2Fframe-removal.html&limit=50000

Diff:
--- /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal-expected.txt
+++ /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal-actual.txt
@@ -1,4 +1,4 @@
   
 
-PASS Frame removal should cancel its revalidations 
+FAIL Frame removal should cancel its revalidations assert_equals: expected "2" but got "1"

[Radar WebKit Bug Importer]

<rdar://problem/60634122>

[Jason Lawrence]

I can reproduce this issue with r258694 using the command below.
run-webkit-tests --iterations 1400 --exit-after-n-failures 3 --force -f --no-retry imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html

[12/1400] imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html failed unexpectedly (text diff)
[32/1400] imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html failed unexpectedly (text diff)
[63/1400] imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html failed unexpectedly (text diff)
Exiting early after 3 failures. 60 tests run.


57 tests ran as expected, 3 didn't (1340 didn't run):

[Jason Lawrence]

I have marked this test as failing while this issue is investigated.
https://trac.webkit.org/changeset/258703/webkit

[Rob Buis]

This should be fixed by https://bugs.webkit.org/show_bug.cgi?id=210905.

[Rob Buis]

Oops, closing wrong bug.

[Rob Buis]

Created attachment 398499 [details]
Patch

[Darin Adler]

Comment on attachment 398499 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=398499&action=review

> LayoutTests/imported/w3c/ChangeLog:15
> +        * web-platform-tests/fetch/stale-while-revalidate/frame-removal.html:
> +        * web-platform-tests/fetch/stale-while-revalidate/resources/stale-frame.py:
> +        (main):

Will this change get upstreamed?

[EWS]

Committed r261455: <https://trac.webkit.org/changeset/261455>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 398499 [details].

[Rob Buis]

Comment on attachment 398499 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=398499&action=review

>> LayoutTests/imported/w3c/ChangeLog:15
>> +        (main):
> 
> Will this change get upstreamed?

I would like to, but Google has some problems with the test related to the chrome implementation, see https://github.com/web-platform-tests/wpt/pull/22292. I'll try to follow the hints in that issue to see if we can get to a test that works for multiple implementations.