Bug 227319

Summary: 5 media/modern-media-controls/* tests are failing constantly on Catalina
Product: WebKit Reporter: Truitt Savell <tsavell>
Component: MediaAssignee: Nobody <webkit-unassigned>
Status: REOPENED ---    
Severity: Normal CC: ap, hi, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch
none
Patch none

Description Truitt Savell 2021-06-23 14:38:33 PDT 
media/modern-media-controls/overflow-support/playback-speed.html [ Failure ]
  media/modern-media-controls/tracks-support/auto-text-track.html [ Failure ]
  media/modern-media-controls/tracks-support/hidden-tracks.html [ Failure ]
  media/modern-media-controls/tracks-support/off-text-track.html [ Failure ]
  media/modern-media-controls/tracks-support/sorted-by-user-preferred-languages.html [ Failure ]

These 5 tests have been failing on Catalina for almost a month. 

History:
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=media%2Fmodern-media-controls%2Foverflow-support%2Fplayback-speed.html&test=media%2Fmodern-media-controls%2Ftracks-support%2Fauto-text-track.html&test=media%2Fmodern-media-controls%2Ftracks-support%2Fhidden-tracks.html&test=media%2Fmodern-media-controls%2Ftracks-support%2Foff-text-track.html&test=media%2Fmodern-media-controls%2Ftracks-support%2Fsorted-by-user-preferred-languages.html

Diff for media/modern-media-controls/tracks-support/auto-text-track.html
--- /Volumes/Data/worker/catalina-release-tests-wk2/build/layout-test-results/media/modern-media-controls/tracks-support/auto-text-track-expected.txt
+++ /Volumes/Data/worker/catalina-release-tests-wk2/build/layout-test-results/media/modern-media-controls/tracks-support/auto-text-track-actual.txt
@@ -8,7 +8,7 @@
 Setting caption display mode to 'automatic'...
 Tapping tracks button...
 PASS checkedItems.length is 1
-PASS checkedItems[0].title is "Auto (Recommended)"
+FAIL checkedItems[0].title should be Auto (Recommended). Was localized string not found.
 Setting caption display mode to 'forcedonly'...
 PASS successfullyParsed is true
 

Results: https://build.webkit.org/results/Apple-Catalina-Release-WK2-Tests/r279179%20(3534)/results.html
Comment 1 Radar WebKit Bug Importer 2021-06-23 14:38:52 PDT 
<rdar://problem/79693462>
Comment 2 Truitt Savell 2021-06-23 14:45:26 PDT 
these three 

  media/modern-media-controls/tracks-support/auto-text-track.html [ Failure ]
  media/modern-media-controls/tracks-support/hidden-tracks.html [ Failure ]
  media/modern-media-controls/tracks-support/off-text-track.html [ Failure ]

began failing around r278207

media/modern-media-controls/overflow-support/playback-speed.html [ Failure ]
has been failing for its whole history 

media/modern-media-controls/tracks-support/sorted-by-user-preferred-languages.html [ Failure ]
has been failing for its whole history
Comment 3 Devin Rousso 2021-06-29 16:03:59 PDT 
Created attachment 432548 [details]
Patch
Comment 4 Devin Rousso 2021-06-29 16:15:47 PDT 
Created attachment 432549 [details]
Patch

don't skip `http/tests/media/modern-media-controls/overflow-support` as that's working just fine

it's very odd that this works but `media/modern-media-controls/overflow-support` doesn't o_0
Comment 5 EWS 2021-06-29 17:14:34 PDT 
Committed r279397 (239262@main): <https://commits.webkit.org/239262@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 432549 [details].
Comment 6 Alexey Proskuryakov 2021-06-29 17:39:48 PDT 
Reopening, as this patch only changed TestExpectations.