222825 2021-03-05 14:00:03 -0800 [Big Sur arm64] TestWebKitAPI.AppleLanguagesTest.UpdateAppleLanguages is timing out 2021-07-21 09:24:41 -0700 1 1 1 Unclassified WebKit New Bugs Other Unspecified Unspecified NEW https://bugs.webkit.org/show_bug.cgi?id=222619 https://bugs.webkit.org/show_bug.cgi?id=222988 https://bugs.webkit.org/show_bug.cgi?id=225429 https://bugs.webkit.org/show_bug.cgi?id=221848 InRadar P2 Normal --- 223037 1 ryanhaddad webkit-unassigned ap cdumez ehutchison pvollan webkit-bot-watchers-bugzilla webkit-bug-importer oldest_to_newest 1736534 0 ryanhaddad 2021-03-05 14:00:03 -0800 TestWebKitAPI.AppleLanguagesTest.UpdateAppleLanguages is timing out on the (new) Big Sur Apple Silicon bots https://build.webkit.org/#/builders/103/builds/19/steps/11/logs/stdio https://results.webkit.org/?suite=api-tests&test=TestWebKitAPI.AppleLanguagesTest.UpdateAppleLanguages 1736535 1 ryanhaddad 2021-03-05 14:00:43 -0800 This test was recently added with https://trac.webkit.org/changeset/273904/webkit, it does not appear to be timing out on the intel-based bots. 1736536 2 webkit-bug-importer 2021-03-05 14:01:34 -0800 <rdar://problem/75110764> 1737931 3 cdumez 2021-03-09 20:54:18 -0800 I have temporarily disabled the test on Apple Silicon in https://commits.webkit.org/235113 because it was causing trouble on the bots. 1738120 4 cdumez 2021-03-10 10:38:45 -0800 ENABLE_CFPREFS_DIRECT_MODE seems to be enabled on Apple Silicon too so I don't see why the preference observer would not observe the AppleLanguages preference changing... 1756688 5 cdumez 2021-05-04 15:30:02 -0700 Tentatively re-enabling the test in http://trac.webkit.org/r276985 to see if the test is still failing and what its output looks like. 1756939 6 cdumez 2021-05-05 07:52:53 -0700 It is still failing, this time with this error: ``` /Volumes/Data/worker/bigsur-release/build/Tools/TestWebKitAPI/Tests/WebKit/OverrideAppleLanguagesPreference.mm:131 Value of: didChangeLanguage Actual: false Expected: true TestWebKitAPI.AppleLanguagesTest.UpdateAppleLanguages Failed ``` 1757005 7 cdumez 2021-05-05 11:34:41 -0700 My bet is that this is a WebContent sandboxing issue. I see in our sandbox profile that some rules are CPU-specific. Sadly, I do not have a M1 to debug this. 1757217 8 cdumez 2021-05-05 18:48:05 -0700 Note that the test is flaky on Apple Silicon rather than 100% failing. As a result, this seems to be a potential timing issue. 1757218 9 cdumez 2021-05-05 18:49:50 -0700 (In reply to Chris Dumez from comment #8) > Note that the test is flaky on Apple Silicon rather than 100% failing. As a > result, this seems to be a potential timing issue. According to the flakiness dashboard, the test is also flaky on Intel. However, due to timing, the test is almost always passing on Intel and almost always failing on Apple Silicon. 1757234 10 cdumez 2021-05-05 20:05:43 -0700 With some extra logging and Alexey's help, I was able to figure out that the WKPreferenceObserver does get constructed WKPreferenceObserver.preferenceDidChange does not get called when I update AppleLanguages. Something seems unreliable with the WKPreferenceObserver. Now that I have found Bug 221848, maybe this is not too surprising. It seems many of the WKPreferenceObserver API tests are currently disabled due to flakiness. 1778441 11 ehutchison 2021-07-21 09:24:41 -0700 This test is now flaky failing on Intel almost always since 7/12/21, r279855.