Bug 225521 - REGRESSION(r277116?): [ macOS Release wk2 ] media/modern-media-controls/pip-support/pip-support-enabled.html (layout-test) is a flaky text failure
Summary: REGRESSION(r277116?): [ macOS Release wk2 ] media/modern-media-controls/pip-s...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-05-07 10:53 PDT by Robert Jenner
Modified: 2021-05-17 18:10 PDT (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Jenner 2021-05-07 10:53:52 PDT
media/modern-media-controls/pip-support/pip-support-enabled.html

is a flaky text failure on macOS Catalina and BigSur Release on wk2. 

HISTORY: 
https://results.webkit.org/?suite=layout-tests&test=media%2Fmodern-media-controls%2Fpip-support%2Fpip-support-enabled.html

TEXT DIFF:
 On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
 
 
-PASS mediaController.controls.pipButton.enabled is true
+FAIL mediaController.controls.pipButton.enabled should be true. Was false.
 PASS mediaController.controls.pipButton.enabled is false
 
 PASS successfullyParsed is true

https://build.webkit.org/results/Apple-BigSur-Release-WK2-Tests/r277172%20(2404)/media/modern-media-controls/pip-support/pip-support-enabled-pretty-diff.html
Comment 1 Radar WebKit Bug Importer 2021-05-14 10:54:24 PDT
<rdar://problem/78024081>
Comment 2 Robert Jenner 2021-05-17 18:08:23 PDT
Was able to reproduce the failing at BigSur Release ToT using the following test:

run-webkit-tests media/modern-media-controls/pip-support/pip-support-enabled.html --iterations 1000 -f

I was able to bisect this down having passed at r277112 and starting to fail at r277116. Looking through all of the commits between r277112 and r277116, it does look like changes at r277116 could potentially have caused this test to start flaky failing:
https://trac.webkit.org/changeset/277116/webkit

I have updated the test expectations to Pass Failure here:
https://trac.webkit.org/changeset/277625/webkit