Bug 196502 - [ Mac WK1 ] REGRESSION (r240537) Layout Test media/video-background-tab-playback.html is a flaky failure
Summary: [ Mac WK1 ] REGRESSION (r240537) Layout Test media/video-background-tab-playb...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Eric Carlson
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2019-04-02 11:31 PDT by Shawn Roberts
Modified: 2020-03-20 13:42 PDT (History)
10 users (show)

See Also:


Attachments
Patch (5.16 KB, patch)
2020-03-19 15:47 PDT, Eric Carlson
no flags Details | Formatted Diff | Diff
Revert changes made to iOS TestExpectations (1.49 KB, patch)
2020-03-20 08:30 PDT, Eric Carlson
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Shawn Roberts 2019-04-02 11:31:31 PDT
The following layout test is flaky on Mac Release WK1

media/video-background-tab-playback.html

Probable cause:

Test is a flaky failure on the dashboard for its entire available history. Local testing found approximately 10 failures in 500 iterations after https://trac.webkit.org/changeset/240537/webkit

Testing done with r240536 and older yield passing results.

Reproduced with:

run-webkit-tests media/video-background-tab-playback.html --iterations 500 -f -1

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=media%2Fvideo-background-tab-playback.html

Diff:

--- /Volumes/Data/slave/mojave-release-tests-wk1/build/layout-test-results/media/video-background-tab-playback-expected.txt
+++ /Volumes/Data/slave/mojave-release-tests-wk1/build/layout-test-results/media/video-background-tab-playback-actual.txt
@@ -13,14 +13,14 @@
 
 
 ** 100ms timer fired...
-EXPECTED (video.paused == 'true') OK
+EXPECTED (video.paused == 'true'), OBSERVED 'false' FAIL
 
 ** Simulate switch back to foreground, video should remain paused.
 RUN(internals.applicationWillEnterForeground())
 
 
 ** 100ms timer fired...
-EXPECTED (video.paused == 'true') OK
+EXPECTED (video.paused == 'true'), OBSERVED 'false' FAIL
 
 END OF TEST
Comment 1 Shawn Roberts 2019-04-02 11:37:14 PDT
Marked flaky in https://trac.webkit.org/changeset/243749/webkit while waiting for a fix.
Comment 2 Radar WebKit Bug Importer 2019-04-02 13:46:39 PDT
<rdar://problem/49532580>
Comment 3 Eric Carlson 2020-03-19 15:47:50 PDT
Created attachment 394039 [details]
Patch
Comment 4 EWS 2020-03-19 17:50:30 PDT
Committed r258747: <https://trac.webkit.org/changeset/258747>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 394039 [details].
Comment 5 Aakash Jain 2020-03-20 07:35:32 PDT
media/video-background-tab-playback.html is failing on iOS since this change.
History: https://results.webkit.org/?suite=layout-tests&test=media%2Fvideo-background-tab-playback.html
Comment 6 Eric Carlson 2020-03-20 08:30:44 PDT
Reopening to attach new patch.
Comment 7 Eric Carlson 2020-03-20 08:30:45 PDT
Created attachment 394084 [details]
Revert changes made to iOS TestExpectations
Comment 8 EWS 2020-03-20 08:51:42 PDT
Committed r258763: <https://trac.webkit.org/changeset/258763>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 394084 [details].