241991 2022-06-24 16:11:57 -0700 REGRESSION (251792@main?): [ macOS ][ iOS ] TestWebKitAPI.WKBackForwardList.BackForwardNavigationDoesNotSkipItemsWithUserGesture is a flaky failure and crash 2022-06-27 16:36:09 -0700 1 1 1 Unclassified WebKit New Bugs WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=241932 InRadar P2 Normal --- 1 rackler cdumez cdumez webkit-bot-watchers-bugzilla webkit-bug-importer oldest_to_newest 1878285 0 rackler 2022-06-24 16:11:57 -0700 Description: TestWebKitAPI.WKBackForwardList.BackForwardNavigationDoesNotSkipItemsWithUserGesture The first failure that I saw on the dashboard was on 6/23/2022 at 251792@main. The test is flaky crashing on iOS arm64 and failing on iOS X86_64. The flaky crashing on iOS arm64 may be bot related because other test are failing similarly. The test is flaky failing on macOS. The flakiness dashboard indicates that 251792@main may be the commit causing the issue. There is a good possibility that the test landed this way and the bots did not show the results until later on the timeline. REPRODUCTION STEPS I can reproduce this on 251793@main, but unable to reproduce it on 251789@main or earlier but, I was not able to find a commit that seemed relevant; due to the lack of ability to test commits in that range, I cannot pinpoint what change could have caused this. I was also able to reproduce on 251793@main and ran again test again and it did not reproduce, which adds to the mystery. Command: run-api-tests --debug --iterations 200 TestWebKitAPI.WKBackForwardList.BackForwardNavigationDoesNotSkipItemsWithUserGesture Result: Failed TestWebKitAPI.WKBackForwardList.BackForwardNavigationDoesNotSkipItemsWithUserGesture /Volumes/Data/worker/Apple-BigSur-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKit/WKBackForwardList.mm:405 Expected equality of these values: [webView URL].absoluteString.UTF8String Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple2.html" url1.absoluteString.UTF8String Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple.html" /Volumes/Data/worker/Apple-BigSur-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKit/WKBackForwardList.mm:410 Expected equality of these values: [webView URL].absoluteString.UTF8String Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple2.html#a" url2.absoluteString.UTF8String Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple2.html" /Volumes/Data/worker/Apple-BigSur-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKit/WKBackForwardList.mm:415 Expected equality of these values: [webView URL].absoluteString.UTF8String Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple.html" lastURL.absoluteString.UTF8String Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple2.html#a" History: https://results.webkit.org/?suite=api-tests&test=TestWebKitAPI.WKBackForwardList.BackForwardNavigationDoesNotSkipItemsWithUserGesture&limit=50000 run-api-tests stdio: Failed TestWebKitAPI.WKBackForwardList.BackForwardNavigationDoesNotSkipItemsWithUserGesture /Volumes/Data/worker/Apple-BigSur-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKit/WKBackForwardList.mm:405 Expected equality of these values: [webView URL].absoluteString.UTF8String Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple2.html" url1.absoluteString.UTF8String Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple.html" /Volumes/Data/worker/Apple-BigSur-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKit/WKBackForwardList.mm:410 Expected equality of these values: [webView URL].absoluteString.UTF8String Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple2.html#a" url2.absoluteString.UTF8String Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple2.html" /Volumes/Data/worker/Apple-BigSur-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKit/WKBackForwardList.mm:415 Expected equality of these values: [webView URL].absoluteString.UTF8String Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple.html" lastURL.absoluteString.UTF8String Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple2.html#a" iOS arm64 results: https://build.webkit.org/#/builders/529/builds/108 TestWebKitAPI.WKBackForwardList.BackForwardNavigationDoesNotSkipItemsWithUserGesture An error was encountered processing the command (domain=NSMachErrorDomain, code=24): The operation couldn’t be completed. (Mach error 24 - (os/kern) memory data moved) 1878286 1 webkit-bug-importer 2022-06-24 16:12:19 -0700 <rdar://problem/95886849> 1878289 2 cdumez 2022-06-24 16:18:20 -0700 Hmm, `run-api-tests --debug --no-build --iterations 200 TestWebKitAPI.WKBackForwardList.BackForwardNavigationDoesNotSkipItemsWithUserGesture` didn't reproduce for me. This test was modified in https://commits.webkit.org/251837@main. I wonder if 251837@main fixed it or if I'm just unable to reproduce due to timing issues. I guess the bots will tell me. 1878291 3 cdumez 2022-06-24 16:25:57 -0700 Ok, I managed to reproduce on an M1 machine. I am investigating. Odds are that the test was already flaky when introduced. 1878298 4 cdumez 2022-06-24 16:56:24 -0700 Pull request: https://github.com/WebKit/WebKit/pull/1789 1878623 5 ews-feeder 2022-06-27 16:36:07 -0700 Committed 251894@main (2a7a51012dfc): <https://commits.webkit.org/251894@main> Reviewed commits have been landed. Closing PR #1789 and removing active labels.