Bug 229832

Summary:	[ Mac EWS ] imported/w3c/web-platform-tests/beacon/beacon-redirect.https.window.html is a flaky failure
Product:	WebKit	Reporter:	ayumi_kojima
Component:	New Bugs	Assignee:	Nobody <webkit-unassigned>
Status:	REOPENED ---
Severity:	Normal	CC:	achristensen, cdumez, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified
See Also:	https://bugs.webkit.org/show_bug.cgi?id=237536

Description ayumi_kojima 2021-09-02 15:17:38 PDT

imported/w3c/web-platform-tests/beacon/beacon-redirect.https.window.html

Is a flaky failure on macOS-AppleSilicon-Big-Sur-Debug-WK2-Tests-EWS.

The flaky failure is seen (though really flaky) in the open source directory: https://results.webkit.org/?suite=layout-tests&test=imported/w3c/web-platform-tests/beacon/beacon-redirect.https.window.html

The test became flaky on EWS at https://ews-build.webkit.org/#/builders/60/builds/7830

Diff:

--- /Volumes/Data/worker/macOS-AppleSilicon-Big-Sur-Debug-WK2-Tests-EWS/build/layout-test-results/imported/w3c/web-platform-tests/beacon/beacon-redirect.https.window-expected.txt
+++ /Volumes/Data/worker/macOS-AppleSilicon-Big-Sur-Debug-WK2-Tests-EWS/build/layout-test-results/imported/w3c/web-platform-tests/beacon/beacon-redirect.https.window-actual.txt
@@ -4,7 +4,7 @@
 PASS cross-origin, CORS-safelisted: status = 307, type = form
 PASS cross-origin, CORS-safelisted: status = 307, type = blob
 PASS cross-origin, CORS-safelisted: status = 308, type = string
-PASS cross-origin, CORS-safelisted: status = 308, type = arraybuffer
+FAIL cross-origin, CORS-safelisted: status = 308, type = arraybuffer assert_true: timeout expected true got false
 PASS cross-origin, CORS-safelisted: status = 308, type = form
 PASS cross-origin, CORS-safelisted: status = 308, type = blob

Comment 1 Radar WebKit Bug Importer 2021-09-02 15:18:01 PDT

<rdar://problem/82695630>

Comment 2 ayumi_kojima 2021-09-02 15:30:08 PDT

Marked expectations: https://trac.webkit.org/changeset/281959/webkit

Comment 3 ayumi_kojima 2021-09-13 15:11:53 PDT

The test timed out locally on BigSur AS instead of failing.The test hanged with --no-timeout.

Diff:

--- /Volumes/Data/Builds/debugToTest/layout-test-results/imported/w3c/web-platform-tests/beacon/beacon-redirect.https.window-expected.txt
+++ /Volumes/Data/Builds/debugToTest/layout-test-results/imported/w3c/web-platform-tests/beacon/beacon-redirect.https.window-actual.txt
@@ -1,10 +1,3 @@
+FAIL: Timed out waiting for notifyDone to be called
 
-PASS cross-origin, CORS-safelisted: status = 307, type = string
-PASS cross-origin, CORS-safelisted: status = 307, type = arraybuffer
-PASS cross-origin, CORS-safelisted: status = 307, type = form
-PASS cross-origin, CORS-safelisted: status = 307, type = blob
-PASS cross-origin, CORS-safelisted: status = 308, type = string
-PASS cross-origin, CORS-safelisted: status = 308, type = arraybuffer
-PASS cross-origin, CORS-safelisted: status = 308, type = form
-PASS cross-origin, CORS-safelisted: status = 308, type = blob

Comment 4 ayumi_kojima 2021-09-14 17:52:53 PDT

I was able to reproduce the failure (with a different diff) locally on Catalina using run-webkit-tests --iterations 500 --exit-after-n-failures 1 --force imported/w3c/web-platform-tests/beacon/beacon-redirect.https.window.html


Diff:

--- /Volumes/Data/Builds/Jazz/buildToTest/layout-test-results/imported/w3c/web-platform-tests/beacon/beacon-redirect.https.window-expected.txt
+++ /Volumes/Data/Builds/Jazz/buildToTest/layout-test-results/imported/w3c/web-platform-tests/beacon/beacon-redirect.https.window-actual.txt
@@ -1,10 +1,12 @@
 
-PASS cross-origin, CORS-safelisted: status = 307, type = string
-PASS cross-origin, CORS-safelisted: status = 307, type = arraybuffer
-PASS cross-origin, CORS-safelisted: status = 307, type = form
-PASS cross-origin, CORS-safelisted: status = 307, type = blob
-PASS cross-origin, CORS-safelisted: status = 308, type = string
-PASS cross-origin, CORS-safelisted: status = 308, type = arraybuffer
-PASS cross-origin, CORS-safelisted: status = 308, type = form
-PASS cross-origin, CORS-safelisted: status = 308, type = blob
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT cross-origin, CORS-safelisted: status = 307, type = string Test timed out
+TIMEOUT cross-origin, CORS-safelisted: status = 307, type = arraybuffer Test timed out
+TIMEOUT cross-origin, CORS-safelisted: status = 307, type = form Test timed out
+TIMEOUT cross-origin, CORS-safelisted: status = 307, type = blob Test timed out
+TIMEOUT cross-origin, CORS-safelisted: status = 308, type = string Test timed out
+TIMEOUT cross-origin, CORS-safelisted: status = 308, type = arraybuffer Test timed out
+TIMEOUT cross-origin, CORS-safelisted: status = 308, type = form Test timed out
+TIMEOUT cross-origin, CORS-safelisted: status = 308, type = blob Test timed out
+

Comment 5 ayumi_kojima 2021-09-15 08:16:10 PDT

Test also failed at https://trac.webkit.org/changeset/279224/webkit

--- /Volumes/Data/Builds/Jazz/test-279224/layout-test-results/imported/w3c/web-platform-tests/beacon/beacon-redirect.https.window-expected.txt
+++ /Volumes/Data/Builds/Jazz/test-279224/layout-test-results/imported/w3c/web-platform-tests/beacon/beacon-redirect.https.window-actual.txt
@@ -1,10 +1,12 @@
 
-PASS cross-origin, CORS-safelisted: status = 307, type = string
-PASS cross-origin, CORS-safelisted: status = 307, type = arraybuffer
-PASS cross-origin, CORS-safelisted: status = 307, type = form
-PASS cross-origin, CORS-safelisted: status = 307, type = blob
-PASS cross-origin, CORS-safelisted: status = 308, type = string
-PASS cross-origin, CORS-safelisted: status = 308, type = arraybuffer
-PASS cross-origin, CORS-safelisted: status = 308, type = form
-PASS cross-origin, CORS-safelisted: status = 308, type = blob
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT cross-origin, CORS-safelisted: status = 307, type = string Test timed out
+TIMEOUT cross-origin, CORS-safelisted: status = 307, type = arraybuffer Test timed out
+TIMEOUT cross-origin, CORS-safelisted: status = 307, type = form Test timed out
+TIMEOUT cross-origin, CORS-safelisted: status = 307, type = blob Test timed out
+TIMEOUT cross-origin, CORS-safelisted: status = 308, type = string Test timed out
+TIMEOUT cross-origin, CORS-safelisted: status = 308, type = arraybuffer Test timed out
+TIMEOUT cross-origin, CORS-safelisted: status = 308, type = form Test timed out
+TIMEOUT cross-origin, CORS-safelisted: status = 308, type = blob Test timed out
+

Comment 6 Chris Dumez 2022-03-08 07:25:22 PST

This test no longer exists.

Comment 7 Chris Dumez 2022-03-08 09:15:03 PST

(In reply to Chris Dumez from comment #6)
> This test no longer exists.

I got confused, it is LayoutTests/imported/w3c/web-platform-tests/beacon/beacon-redirect.sub.window.html that is gone. The name is very similar but not quite the same.