Bug 222825 - [Big Sur arm64] TestWebKitAPI.AppleLanguagesTest.UpdateAppleLanguages is timing out
Summary: [Big Sur arm64] TestWebKitAPI.AppleLanguagesTest.UpdateAppleLanguages is timi...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on: 223037
Blocks:
  Show dependency tree / graph
 
Reported: 2021-03-05 14:00 PST by Ryan Haddad
Modified: 2021-07-21 09:24 PDT (History)
6 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Ryan Haddad 2021-03-05 14:00:03 PST 
TestWebKitAPI.AppleLanguagesTest.UpdateAppleLanguages is timing out on the (new) Big Sur Apple Silicon bots

https://build.webkit.org/#/builders/103/builds/19/steps/11/logs/stdio

https://results.webkit.org/?suite=api-tests&test=TestWebKitAPI.AppleLanguagesTest.UpdateAppleLanguages
Comment 1 Ryan Haddad 2021-03-05 14:00:43 PST 
This test was recently added with https://trac.webkit.org/changeset/273904/webkit, it does not appear to be timing out on the intel-based bots.
Comment 2 Radar WebKit Bug Importer 2021-03-05 14:01:34 PST 
<rdar://problem/75110764>
Comment 3 Chris Dumez 2021-03-09 20:54:18 PST 
I have temporarily disabled the test on Apple Silicon in https://commits.webkit.org/235113 because it was causing trouble on the bots.
Comment 4 Chris Dumez 2021-03-10 10:38:45 PST 
ENABLE_CFPREFS_DIRECT_MODE seems to be enabled on Apple Silicon too so I don't see why the preference observer would not observe the AppleLanguages preference changing...
Comment 5 Chris Dumez 2021-05-04 15:30:02 PDT 
Tentatively re-enabling the test in http://trac.webkit.org/r276985 to see if the test is still failing and what its output looks like.
Comment 6 Chris Dumez 2021-05-05 07:52:53 PDT 
It is still failing, this time with this error:
```
    /Volumes/Data/worker/bigsur-release/build/Tools/TestWebKitAPI/Tests/WebKit/OverrideAppleLanguagesPreference.mm:131
    Value of: didChangeLanguage
      Actual: false
    Expected: true
    
TestWebKitAPI.AppleLanguagesTest.UpdateAppleLanguages Failed
```
Comment 7 Chris Dumez 2021-05-05 11:34:41 PDT 
My bet is that this is a WebContent sandboxing issue. I see in our sandbox profile that some rules are CPU-specific. Sadly, I do not have a M1 to debug this.
Comment 8 Chris Dumez 2021-05-05 18:48:05 PDT 
Note that the test is flaky on Apple Silicon rather than 100% failing. As a result, this seems to be a potential timing issue.
Comment 9 Chris Dumez 2021-05-05 18:49:50 PDT 
(In reply to Chris Dumez from comment #8)
> Note that the test is flaky on Apple Silicon rather than 100% failing. As a
> result, this seems to be a potential timing issue.

According to the flakiness dashboard, the test is also flaky on Intel. However, due to timing, the test is almost always passing on Intel and almost always failing on Apple Silicon.
Comment 10 Chris Dumez 2021-05-05 20:05:43 PDT 
With some extra logging and Alexey's help, I was able to figure out that the WKPreferenceObserver does get constructed WKPreferenceObserver.preferenceDidChange does not get called when I update AppleLanguages.

Something seems unreliable with the WKPreferenceObserver. Now that I have found Bug 221848, maybe this is not too surprising. It seems many of the WKPreferenceObserver API tests are currently disabled due to flakiness.
Comment 11 Eric Hutchison 2021-07-21 09:24:41 PDT 
This test is now flaky failing on Intel almost always since 7/12/21, r279855.