Bug 173608

Summary:	Layout Test webrtc/video-replace-muted-track.html is flaky
Product:	WebKit	Reporter:	Matt Lewis <jlewis3>
Component:	New Bugs	Assignee:	Nobody <webkit-unassigned>
Status:	NEW
Severity:	Normal	CC:	ryanhaddad, webkit-bug-importer, youennf
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified
See Also:	https://bugs.webkit.org/show_bug.cgi?id=173486 https://bugs.webkit.org/show_bug.cgi?id=207941

Matt Lewis

Reported 2017-06-20 11:27:56 PDT

webrtc/video-replace-muted-track.html is a flaky failure on iOS simulator according to the flakiness Dashboard. https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=webrtc%2Fvideo-replace-muted-track.html It was added in r218426 and looks to have been flaky since being added. https://trac.webkit.org/changeset/218426/webkit https://build.webkit.org/results/Apple%20iOS%2010%20Simulator%20Release%20WK2%20(Tests)/r218595%20(2415)/results.html https://build.webkit.org/builders/Apple%20iOS%2010%20Simulator%20Release%20WK2%20(Tests)/builds/2415 Diff: --- /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test-results/webrtc/video-replace-muted-track-expected.txt +++ /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test-results/webrtc/video-replace-muted-track-actual.txt @@ -1,3 +1,3 @@ -PASS Switching from disabled to enabled track +FAIL Switching from disabled to enabled track assert_unreached: Reached unreachable code

Attachments
Add attachment proposed patch, testcase, etc.

Matt Lewis

Comment 1 2017-06-20 17:03:30 PDT

Marked test as flaky on iOS WK2: https://trac.webkit.org/changeset/218617/webkit

Radar WebKit Bug Importer

Comment 2 2017-06-22 14:02:52 PDT

<rdar://problem/32932337>

Matt Lewis

Comment 3 2017-06-22 14:17:22 PDT

This test has started to fail on Sierra WK2 and El Capitan WK2 Release. On iOS the test fails far more than passes. https://build.webkit.org/results/Apple%20Sierra%20Debug%20WK2%20(Tests)/r218708%20(1699)/results.html https://build.webkit.org/builders/Apple%20Sierra%20Debug%20WK2%20(Tests)/builds/1699

Matt Lewis

Comment 4 2017-06-23 15:51:58 PDT

Updated test expectations: https://trac.webkit.org/changeset/218770/webkit

Matt Lewis

Comment 5 2017-06-27 14:33:01 PDT

As of this build and revision the Test has stopped failing on iOS Simulator debug, but has started to timeout at 35 seconds. https://trac.webkit.org/changeset/218795/webkit https://build.webkit.org/builders/Apple%20iOS%2010%20Simulator%20Debug%20WK2%20(Tests)/builds/2325

Ryan Haddad

Comment 6 2017-06-28 14:23:27 PDT

Skipped test on iOS in https://trac.webkit.org/changeset/218893

Matt Lewis

Comment 7 2017-07-07 13:39:38 PDT

Adjusted test expectations: https://trac.webkit.org/changeset/219267/webkit

Matt Lewis

Comment 8 2017-10-26 11:08:38 PDT

This test had expectations removed in https://trac.webkit.org/changeset/222273/webkit however the test is still exhibiting the flaky timeout. https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#tests=webrtc%2Fvideo-replace-muted-track.html

youenn fablet

Comment 9 2017-10-26 11:15:36 PDT

Looking at the dashboard, tests sometimes time out for HighSierra and iOS simulator, Release/Debug. They do not time out for Sierra and ElCapitan. I wonder whether the difference is OS related or bot-setup related.

Matt Lewis

Comment 10 2017-10-26 11:17:52 PDT

Skipped in https://trac.webkit.org/changeset/224028/webkit

Note You need to log in before you can comment on or make changes to this bug.