242128 2022-06-29 12:53:38 -0700 REGRESSION (251741@main-251739@main?): [ iOS ] TestWebKitAPI.AutocorrectionTests.AutocorrectionContextBeforeAndAfterEditing is a flaky failure 2022-08-09 01:51:13 -0700 1 1 1 Unclassified WebKit New Bugs WebKit Nightly Build Unspecified Unspecified NEW https://bugs.webkit.org/show_bug.cgi?id=241890 InRadar P2 Normal --- 1 rackler webkit-unassigned ap honamluxurychef ryanhaddad webkit-bot-watchers-bugzilla webkit-bug-importer wenson_hsieh oldest_to_newest 1879092 0 rackler 2022-06-29 12:53:38 -0700 Description: TestWebKitAPI.AutocorrectionTests.AutocorrectionContextBeforeAndAfterEditing The first failure that I saw on the dashboard was on 6/22/2022 at 251749@main. REPRODUCTION STEPS I can reproduce this on 251741@main, but unable to reproduce it on 251738@main or earlier. This affects the iOS model iPhone 8 only. Command: run-api-tests --release --verbose --iterations 10 --model iPhone8 TestWebKitAPI.AutocorrectionTests.AutocorrectionContextBeforeAndAfterEditing Result: Same as bot below History: https://results.webkit.org/?suite=api-tests&test=TestWebKitAPI.AutocorrectionTests.AutocorrectionContextBeforeAndAfterEditing&limit=50000&platform=ios&model=iPhone%208 run-api-tests stdio: Failed TestWebKitAPI.AutocorrectionTests.AutocorrectionContextBeforeAndAfterEditing /Volumes/Data/worker/Apple-iOS-15-Simulator-Release-Build/build/Tools/TestWebKitAPI/Tests/ios/AutocorrectionTestsIOS.mm:182 Expected equality of these values: "hello" [contextAfterInsertingText contextBeforeSelection] Which is: "" /Volumes/Data/worker/Apple-iOS-15-Simulator-Release-Build/build/Tools/TestWebKitAPI/Tests/ios/AutocorrectionTestsIOS.mm:187 Expected equality of these values: "hello" [contextAfterSelecting selectedText] Which is: "" 1879094 1 webkit-bug-importer 2022-06-29 12:53:56 -0700 <rdar://problem/96161911> 1879103 2 rackler 2022-06-29 13:03:04 -0700 I have disabled this test while this issue is investigated. 1879106 3 ews-feeder 2022-06-29 13:11:29 -0700 Test gardening commit 251961@main (9aa946bf2540): <https://commits.webkit.org/251961@main> Reviewed commits have been landed. Closing PR #1912 and removing active labels. 1879418 4 rackler 2022-06-30 10:28:07 -0700 The test failure is referencing contextAfterInsertingText and contextAfterSelecting from 246659@main. The flakiness dashboard shows a clear change from consistently passing to flaky failure first appearing on the dashboard at 251749@main. Reproduction steps further reduced the range(251741@main-251739@main), but it is unclear what in this range may have caused this. 1879470 5 wenson_hsieh 2022-06-30 12:38:19 -0700 (In reply to Karl Rackler from comment #4) > The test failure is referencing contextAfterInsertingText and > contextAfterSelecting from 246659@main. > > The flakiness dashboard shows a clear change from consistently passing to > flaky failure first appearing on the dashboard at 251749@main. > > Reproduction steps further reduced the range(251741@main-251739@main), but > it is unclear what in this range may have caused this. Yeah, neither of the commits in that range remotely seem like they could've caused these test failures. Were there any interesting configuration changes might've affected the bots at around the same time? (i.e. upgrading to a different version, or using new simulator device models, etc.) 1880092 6 ryanhaddad 2022-07-01 12:58:41 -0700 (In reply to Wenson Hsieh from comment #5) > Were there any interesting configuration changes might've affected the bots > at around the same time? (i.e. upgrading to a different version, or using > new simulator device models, etc.) We have not made any changes in this area recently, we're also seeing this failure on internal bots in roughly the same regression range http://commits.webkit.org/compare/251744@main...251736@main 1880094 7 ryanhaddad 2022-07-01 12:59:52 -0700 Not sure what happened with my URL in comment 6, it should be showing a range of 251744@main - 251736@main http://commits.webkit.org/compare/251744@main...251736@main