[ iOS 15 Debug ] imported/w3c/web-platform-tests/xhr/send-timeout-events.htm is a flaky failure

• Component: New Bugs
• Status: NEW ---

imported/w3c/web-platform-tests/xhr/send-timeout-events.htm

Is flaky failing on iOS 15 Debug.

History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fxhr%2Fsend-timeout-events.htm

Diff:
--- /Volumes/Data/worker/ios-simulator-15-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/xhr/send-timeout-events-expected.txt
+++ /Volumes/Data/worker/ios-simulator-15-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/xhr/send-timeout-events-actual.txt
@@ -1,3 +1,3 @@
 
-PASS XMLHttpRequest: The send() method: timeout is not 0
+FAIL XMLHttpRequest: The send() method: timeout is not 0  assert_equals: expected 0 but got 857098

[Radar WebKit Bug Importer]

<rdar://problem/83956855>

Marked test expectations https://trac.webkit.org/changeset/283677/webkit

I was able to reproduce the failure locally on iOS 15 using run-webkit-tests --force --iterations 500 --exit-after-n-failures 1 --debug imported/w3c/web-platform-tests/xhr/send-timeout-events.htm --ios-simulator --no-timeout.

The test failed with r269120 and passed with r269118 (r269119 is not available to test).

[Jonathan Bedard]

(In reply to ayumi_kojima from comment #3)
> I was able to reproduce the failure locally on iOS 15 using run-webkit-tests
> --force --iterations 500 --exit-after-n-failures 1 --debug
> imported/w3c/web-platform-tests/xhr/send-timeout-events.htm --ios-simulator
> --no-timeout.
> 
> The test failed with r269120 and passed with r269118 (r269119 is not
> available to test).

r269120 definitely can't cause a layout test failure like this, it didn't actually change anything about the binary you're testing.

I worked on reproduction again just to make sure. Test failed with r269121 and r269120 and passed with r269118 and r269117. So most likely it is either r269120 or r269119.

[Ryan Haddad]

r269119 was a change on a branch, so it couldn't have affected trunk.

r269120 was a rename of a directory used by buildbot, so it is also unlikely to have impacted test results.

In spite of the reproduction, I still find this hard to believe.