Bug 209281 - [ iOS wk2 and Mac wk2 ] imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html is flaky failing.
Summary: [ iOS wk2 and Mac wk2 ] imported/w3c/web-platform-tests/fetch/stale-while-rev...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Mac macOS 10.14
: P2 Normal
Assignee: Rob Buis
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-03-19 08:15 PDT by Jason Lawrence
Modified: 2020-05-11 00:40 PDT (History)
4 users (show)

See Also:


Attachments
Patch (4.00 KB, patch)
2020-05-05 05:43 PDT, Rob Buis
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jason Lawrence 2020-03-19 08:15:38 PDT
imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html

Description:
This test is flaky failing on Mac wk2 and iOS wk2. The flaky failures are visible since the introduction of the test.

History:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Ffetch%2Fstale-while-revalidate%2Fframe-removal.html&limit=50000

Diff:
--- /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal-expected.txt
+++ /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal-actual.txt
@@ -1,4 +1,4 @@
   
 
-PASS Frame removal should cancel its revalidations 
+FAIL Frame removal should cancel its revalidations assert_equals: expected "2" but got "1"
Comment 1 Radar WebKit Bug Importer 2020-03-19 08:16:22 PDT
<rdar://problem/60634122>
Comment 2 Jason Lawrence 2020-03-19 08:29:58 PDT
I can reproduce this issue with r258694 using the command below.
run-webkit-tests --iterations 1400 --exit-after-n-failures 3 --force -f --no-retry imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html

[12/1400] imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html failed unexpectedly (text diff)
[32/1400] imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html failed unexpectedly (text diff)
[63/1400] imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html failed unexpectedly (text diff)
Exiting early after 3 failures. 60 tests run.


57 tests ran as expected, 3 didn't (1340 didn't run):
Comment 3 Jason Lawrence 2020-03-19 09:51:16 PDT
I have marked this test as failing while this issue is investigated.
https://trac.webkit.org/changeset/258703/webkit
Comment 4 Rob Buis 2020-04-24 02:34:40 PDT
This should be fixed by https://bugs.webkit.org/show_bug.cgi?id=210905.
Comment 5 Rob Buis 2020-04-24 02:35:02 PDT
Oops, closing wrong bug.
Comment 6 Rob Buis 2020-05-05 05:43:40 PDT
Created attachment 398499 [details]
Patch
Comment 7 Darin Adler 2020-05-09 17:13:12 PDT
Comment on attachment 398499 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=398499&action=review

> LayoutTests/imported/w3c/ChangeLog:15
> +        * web-platform-tests/fetch/stale-while-revalidate/frame-removal.html:
> +        * web-platform-tests/fetch/stale-while-revalidate/resources/stale-frame.py:
> +        (main):

Will this change get upstreamed?
Comment 8 EWS 2020-05-10 00:26:20 PDT
Committed r261455: <https://trac.webkit.org/changeset/261455>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 398499 [details].
Comment 9 Rob Buis 2020-05-11 00:40:48 PDT
Comment on attachment 398499 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=398499&action=review

>> LayoutTests/imported/w3c/ChangeLog:15
>> +        (main):
> 
> Will this change get upstreamed?

I would like to, but Google has some problems with the test related to the chrome implementation, see https://github.com/web-platform-tests/wpt/pull/22292. I'll try to follow the hints in that issue to see if we can get to a test that works for multiple implementations.