Bug 227319 - 5 media/modern-media-controls/* tests are failing constantly on Catalina
Summary: 5 media/modern-media-controls/* tests are failing constantly on Catalina
Status: REOPENED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-06-23 14:38 PDT by Truitt Savell
Modified: 2021-12-20 11:15 PST (History)
4 users (show)

See Also:


Attachments
Patch (1.57 KB, patch)
2021-06-29 16:03 PDT, Devin Rousso
no flags Details | Formatted Diff | Diff
Patch (1.50 KB, patch)
2021-06-29 16:15 PDT, Devin Rousso
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Truitt Savell 2021-06-23 14:38:33 PDT
media/modern-media-controls/overflow-support/playback-speed.html [ Failure ]
  media/modern-media-controls/tracks-support/auto-text-track.html [ Failure ]
  media/modern-media-controls/tracks-support/hidden-tracks.html [ Failure ]
  media/modern-media-controls/tracks-support/off-text-track.html [ Failure ]
  media/modern-media-controls/tracks-support/sorted-by-user-preferred-languages.html [ Failure ]

These 5 tests have been failing on Catalina for almost a month. 

History:
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=media%2Fmodern-media-controls%2Foverflow-support%2Fplayback-speed.html&test=media%2Fmodern-media-controls%2Ftracks-support%2Fauto-text-track.html&test=media%2Fmodern-media-controls%2Ftracks-support%2Fhidden-tracks.html&test=media%2Fmodern-media-controls%2Ftracks-support%2Foff-text-track.html&test=media%2Fmodern-media-controls%2Ftracks-support%2Fsorted-by-user-preferred-languages.html

Diff for media/modern-media-controls/tracks-support/auto-text-track.html
--- /Volumes/Data/worker/catalina-release-tests-wk2/build/layout-test-results/media/modern-media-controls/tracks-support/auto-text-track-expected.txt
+++ /Volumes/Data/worker/catalina-release-tests-wk2/build/layout-test-results/media/modern-media-controls/tracks-support/auto-text-track-actual.txt
@@ -8,7 +8,7 @@
 Setting caption display mode to 'automatic'...
 Tapping tracks button...
 PASS checkedItems.length is 1
-PASS checkedItems[0].title is "Auto (Recommended)"
+FAIL checkedItems[0].title should be Auto (Recommended). Was localized string not found.
 Setting caption display mode to 'forcedonly'...
 PASS successfullyParsed is true
 

Results: https://build.webkit.org/results/Apple-Catalina-Release-WK2-Tests/r279179%20(3534)/results.html
Comment 1 Radar WebKit Bug Importer 2021-06-23 14:38:52 PDT
<rdar://problem/79693462>
Comment 2 Truitt Savell 2021-06-23 14:45:26 PDT
these three 

  media/modern-media-controls/tracks-support/auto-text-track.html [ Failure ]
  media/modern-media-controls/tracks-support/hidden-tracks.html [ Failure ]
  media/modern-media-controls/tracks-support/off-text-track.html [ Failure ]

began failing around r278207

media/modern-media-controls/overflow-support/playback-speed.html [ Failure ]
has been failing for its whole history 

media/modern-media-controls/tracks-support/sorted-by-user-preferred-languages.html [ Failure ]
has been failing for its whole history
Comment 3 Devin Rousso 2021-06-29 16:03:59 PDT
Created attachment 432548 [details]
Patch
Comment 4 Devin Rousso 2021-06-29 16:15:47 PDT
Created attachment 432549 [details]
Patch

don't skip `http/tests/media/modern-media-controls/overflow-support` as that's working just fine

it's very odd that this works but `media/modern-media-controls/overflow-support` doesn't o_0
Comment 5 EWS 2021-06-29 17:14:34 PDT
Committed r279397 (239262@main): <https://commits.webkit.org/239262@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 432549 [details].
Comment 6 Alexey Proskuryakov 2021-06-29 17:39:48 PDT
Reopening, as this patch only changed TestExpectations.