Bug 231337 - [ iOS 15 Debug ] imported/w3c/web-platform-tests/xhr/send-timeout-events.htm is a flaky failure
Summary: [ iOS 15 Debug ] imported/w3c/web-platform-tests/xhr/send-timeout-events.htm ...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: iPhone / iPad Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-10-06 16:11 PDT by ayumi_kojima
Modified: 2021-11-04 10:19 PDT (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description ayumi_kojima 2021-10-06 16:11:42 PDT
imported/w3c/web-platform-tests/xhr/send-timeout-events.htm

Is flaky failing on iOS 15 Debug.

History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fxhr%2Fsend-timeout-events.htm

Diff:
--- /Volumes/Data/worker/ios-simulator-15-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/xhr/send-timeout-events-expected.txt
+++ /Volumes/Data/worker/ios-simulator-15-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/xhr/send-timeout-events-actual.txt
@@ -1,3 +1,3 @@
 
-PASS XMLHttpRequest: The send() method: timeout is not 0
+FAIL XMLHttpRequest: The send() method: timeout is not 0  assert_equals: expected 0 but got 857098
Comment 1 Radar WebKit Bug Importer 2021-10-06 16:12:42 PDT
<rdar://problem/83956855>
Comment 2 ayumi_kojima 2021-10-06 16:16:20 PDT
Marked test expectations https://trac.webkit.org/changeset/283677/webkit
Comment 3 ayumi_kojima 2021-10-20 14:37:36 PDT
I was able to reproduce the failure locally on iOS 15 using run-webkit-tests --force --iterations 500 --exit-after-n-failures 1 --debug imported/w3c/web-platform-tests/xhr/send-timeout-events.htm --ios-simulator --no-timeout.

The test failed with r269120 and passed with r269118 (r269119 is not available to test).
Comment 4 Jonathan Bedard 2021-10-21 08:03:04 PDT
(In reply to ayumi_kojima from comment #3)
> I was able to reproduce the failure locally on iOS 15 using run-webkit-tests
> --force --iterations 500 --exit-after-n-failures 1 --debug
> imported/w3c/web-platform-tests/xhr/send-timeout-events.htm --ios-simulator
> --no-timeout.
> 
> The test failed with r269120 and passed with r269118 (r269119 is not
> available to test).

r269120 definitely can't cause a layout test failure like this, it didn't actually change anything about the binary you're testing.
Comment 5 ayumi_kojima 2021-11-03 12:02:19 PDT
I worked on reproduction again just to make sure. Test failed with r269121 and r269120 and passed with r269118 and r269117. So most likely it is either r269120 or r269119.
Comment 6 Ryan Haddad 2021-11-03 12:11:24 PDT
r269119 was a change on a branch, so it couldn't have affected trunk.

r269120 was a rename of a directory used by buildbot, so it is also unlikely to have impacted test results.

In spite of the reproduction, I still find this hard to believe.