Bug 196502

Summary: [ Mac WK1 ] REGRESSION (r240537) Layout Test media/video-background-tab-playback.html is a flaky failure
Product: WebKit Reporter: Shawn Roberts <sroberts>
Component: Tools / TestsAssignee: Eric Carlson <eric.carlson>
Status: RESOLVED FIXED    
Severity: Normal CC: aakash_jain, eric.carlson, ews-watchlist, glenn, jer.noble, lforschler, philipj, sergio, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=209353
Attachments:
Description Flags
Patch
none
Revert changes made to iOS TestExpectations none

Description Shawn Roberts 2019-04-02 11:31:31 PDT 
The following layout test is flaky on Mac Release WK1

media/video-background-tab-playback.html

Probable cause:

Test is a flaky failure on the dashboard for its entire available history. Local testing found approximately 10 failures in 500 iterations after https://trac.webkit.org/changeset/240537/webkit

Testing done with r240536 and older yield passing results.

Reproduced with:

run-webkit-tests media/video-background-tab-playback.html --iterations 500 -f -1

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=media%2Fvideo-background-tab-playback.html

Diff:

--- /Volumes/Data/slave/mojave-release-tests-wk1/build/layout-test-results/media/video-background-tab-playback-expected.txt
+++ /Volumes/Data/slave/mojave-release-tests-wk1/build/layout-test-results/media/video-background-tab-playback-actual.txt
@@ -13,14 +13,14 @@
 
 
 ** 100ms timer fired...
-EXPECTED (video.paused == 'true') OK
+EXPECTED (video.paused == 'true'), OBSERVED 'false' FAIL
 
 ** Simulate switch back to foreground, video should remain paused.
 RUN(internals.applicationWillEnterForeground())
 
 
 ** 100ms timer fired...
-EXPECTED (video.paused == 'true') OK
+EXPECTED (video.paused == 'true'), OBSERVED 'false' FAIL
 
 END OF TEST
Comment 1 Shawn Roberts 2019-04-02 11:37:14 PDT 
Marked flaky in https://trac.webkit.org/changeset/243749/webkit while waiting for a fix.
Comment 2 Radar WebKit Bug Importer 2019-04-02 13:46:39 PDT 
<rdar://problem/49532580>
Comment 3 Eric Carlson 2020-03-19 15:47:50 PDT 
Created attachment 394039 [details]
Patch
Comment 4 EWS 2020-03-19 17:50:30 PDT 
Committed r258747: <https://trac.webkit.org/changeset/258747>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 394039 [details].
Comment 5 Aakash Jain 2020-03-20 07:35:32 PDT 
media/video-background-tab-playback.html is failing on iOS since this change.
History: https://results.webkit.org/?suite=layout-tests&test=media%2Fvideo-background-tab-playback.html
Comment 6 Eric Carlson 2020-03-20 08:30:44 PDT 
Reopening to attach new patch.
Comment 7 Eric Carlson 2020-03-20 08:30:45 PDT 
Created attachment 394084 [details]
Revert changes made to iOS TestExpectations
Comment 8 EWS 2020-03-20 08:51:42 PDT 
Committed r258763: <https://trac.webkit.org/changeset/258763>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 394084 [details].