Bug 196400 – [ Mac WK2 ] Layout Test fast/mediastream/MediaStreamTrack-getSettings.html is a flaky failure

• Component: WebRTC
• Status: RESOLVED FIXED

[Shawn Roberts]

The following layout test is flaky on Mac WK2 Release and Debug

fast/mediastream/MediaStreamTrack-getSettings.html

Probable cause:

Test has been flaky on the dashboard for the entire available history. 

After changes in https://trac.webkit.org/changeset/243094/webkit I was able to reproduce locally.

Testing locally I got it to fail consistently up until r243094. Prior to 243092 it passes locally, but obviously it fails on the bots prior to the change. I don't know if the timing changes in 243094 have allowed the test to fail locally.

Reproduced with:

run-webkit-tests fast/mediastream/MediaStreamTrack-getSettings.html --iterations 3000 -fg

Using -fg it has about a 15% failure rate locally. If I do it with just 1 child process and -g it fails about 2 in 3000 iterations. 

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=fast%2Fmediastream%2FMediaStreamTrack-getSettings.html

Diff:

--- /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/fast/mediastream/MediaStreamTrack-getSettings-expected.txt
+++ /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/fast/mediastream/MediaStreamTrack-getSettings-actual.txt
@@ -13,7 +13,7 @@
 audio track settings:
   settings.deviceId = <UUID>
   settings.echoCancellation = false
-  settings.sampleRate = 44100
+  settings.sampleRate = 0
   settings.volume = 1
 
 According to the spec: "[every setting] MUST be a member of the set defined for that property by getCapabilities()"

[Shawn Roberts]

Adding Chris Dumez, incase the timing changes affected it at all.

Adding Eric Carlson, as they created the test.

[Shawn Roberts]

marked flaky in https://trac.webkit.org/changeset/243652/webkit while waiting for a fix.

[Radar WebKit Bug Importer]

<rdar://problem/49567579>

[Chris Dumez]

https://trac.webkit.org/changeset/243094/webkit merely impacted timing. This is an example of a flaky test becoming flakier.

[Shawn Roberts]

(In reply to Chris Dumez from comment #4)
> https://trac.webkit.org/changeset/243094/webkit merely impacted timing. This
> is an example of a flaky test becoming flakier.

Correct, I was just pointing out after your revision I was able to reproduce it locally and more consistently. It has been flaky it's available history. I was not blaming you for general flaky behavior.

[Eric Carlson]

Created attachment 380456 [details]
Patch

[Eric Carlson]

The mac-wk1 failure is unrelated.

[WebKit Commit Bot]

Comment on attachment 380456 [details]
Patch

Clearing flags on attachment: 380456

Committed r250918: <https://trac.webkit.org/changeset/250918>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.