Description: TestWebKitAPI.AutocorrectionTests.AutocorrectionContextBeforeAndAfterEditing The first failure that I saw on the dashboard was on 6/22/2022 at 251749@main. REPRODUCTION STEPS I can reproduce this on 251741@main, but unable to reproduce it on 251738@main or earlier. This affects the iOS model iPhone 8 only. Command: run-api-tests --release --verbose --iterations 10 --model iPhone8 TestWebKitAPI.AutocorrectionTests.AutocorrectionContextBeforeAndAfterEditing Result: Same as bot below History: https://results.webkit.org/?suite=api-tests&test=TestWebKitAPI.AutocorrectionTests.AutocorrectionContextBeforeAndAfterEditing&limit=50000&platform=ios&model=iPhone%208 run-api-tests stdio: Failed TestWebKitAPI.AutocorrectionTests.AutocorrectionContextBeforeAndAfterEditing /Volumes/Data/worker/Apple-iOS-15-Simulator-Release-Build/build/Tools/TestWebKitAPI/Tests/ios/AutocorrectionTestsIOS.mm:182 Expected equality of these values: "hello" [contextAfterInsertingText contextBeforeSelection] Which is: "" /Volumes/Data/worker/Apple-iOS-15-Simulator-Release-Build/build/Tools/TestWebKitAPI/Tests/ios/AutocorrectionTestsIOS.mm:187 Expected equality of these values: "hello" [contextAfterSelecting selectedText] Which is: ""
<rdar://problem/96161911>
I have disabled this test while this issue is investigated.
Test gardening commit 251961@main (9aa946bf2540): <https://commits.webkit.org/251961@main> Reviewed commits have been landed. Closing PR #1912 and removing active labels.
The test failure is referencing contextAfterInsertingText and contextAfterSelecting from 246659@main. The flakiness dashboard shows a clear change from consistently passing to flaky failure first appearing on the dashboard at 251749@main. Reproduction steps further reduced the range(251741@main-251739@main), but it is unclear what in this range may have caused this.
(In reply to Karl Rackler from comment #4) > The test failure is referencing contextAfterInsertingText and > contextAfterSelecting from 246659@main. > > The flakiness dashboard shows a clear change from consistently passing to > flaky failure first appearing on the dashboard at 251749@main. > > Reproduction steps further reduced the range(251741@main-251739@main), but > it is unclear what in this range may have caused this. Yeah, neither of the commits in that range remotely seem like they could've caused these test failures. Were there any interesting configuration changes might've affected the bots at around the same time? (i.e. upgrading to a different version, or using new simulator device models, etc.)
(In reply to Wenson Hsieh from comment #5) > Were there any interesting configuration changes might've affected the bots > at around the same time? (i.e. upgrading to a different version, or using > new simulator device models, etc.) We have not made any changes in this area recently, we're also seeing this failure on internal bots in roughly the same regression range http://commits.webkit.org/compare/251744@main...251736@main
Not sure what happened with my URL in comment 6, it should be showing a range of 251744@main - 251736@main http://commits.webkit.org/compare/251744@main...251736@main