Bug 196400 - [ Mac WK2 ] Layout Test fast/mediastream/MediaStreamTrack-getSettings.html is a flaky failure
Summary: [ Mac WK2 ] Layout Test fast/mediastream/MediaStreamTrack-getSettings.html is...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebRTC (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Eric Carlson
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2019-03-29 10:32 PDT by Shawn Roberts
Modified: 2019-10-09 11:10 PDT (History)
15 users (show)

See Also:


Attachments
Patch (10.94 KB, patch)
2019-10-08 13:18 PDT, Eric Carlson
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Shawn Roberts 2019-03-29 10:32:24 PDT
The following layout test is flaky on Mac WK2 Release and Debug

fast/mediastream/MediaStreamTrack-getSettings.html

Probable cause:

Test has been flaky on the dashboard for the entire available history. 

After changes in https://trac.webkit.org/changeset/243094/webkit I was able to reproduce locally.

Testing locally I got it to fail consistently up until r243094. Prior to 243092 it passes locally, but obviously it fails on the bots prior to the change. I don't know if the timing changes in 243094 have allowed the test to fail locally.

Reproduced with:

run-webkit-tests fast/mediastream/MediaStreamTrack-getSettings.html --iterations 3000 -fg

Using -fg it has about a 15% failure rate locally. If I do it with just 1 child process and -g it fails about 2 in 3000 iterations. 

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=fast%2Fmediastream%2FMediaStreamTrack-getSettings.html

Diff:

--- /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/fast/mediastream/MediaStreamTrack-getSettings-expected.txt
+++ /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/fast/mediastream/MediaStreamTrack-getSettings-actual.txt
@@ -13,7 +13,7 @@
 audio track settings:
   settings.deviceId = <UUID>
   settings.echoCancellation = false
-  settings.sampleRate = 44100
+  settings.sampleRate = 0
   settings.volume = 1
 
 According to the spec: "[every setting] MUST be a member of the set defined for that property by getCapabilities()"
Comment 1 Shawn Roberts 2019-03-29 10:34:58 PDT
Adding Chris Dumez, incase the timing changes affected it at all.

Adding Eric Carlson, as they created the test.
Comment 2 Shawn Roberts 2019-03-29 11:37:23 PDT
marked flaky in https://trac.webkit.org/changeset/243652/webkit while waiting for a fix.
Comment 3 Radar WebKit Bug Importer 2019-04-03 12:02:06 PDT
<rdar://problem/49567579>
Comment 4 Chris Dumez 2019-04-03 12:04:48 PDT
https://trac.webkit.org/changeset/243094/webkit merely impacted timing. This is an example of a flaky test becoming flakier.
Comment 5 Shawn Roberts 2019-04-03 12:13:39 PDT
(In reply to Chris Dumez from comment #4)
> https://trac.webkit.org/changeset/243094/webkit merely impacted timing. This
> is an example of a flaky test becoming flakier.

Correct, I was just pointing out after your revision I was able to reproduce it locally and more consistently. It has been flaky it's available history. I was not blaming you for general flaky behavior.
Comment 6 Eric Carlson 2019-10-08 13:18:38 PDT
Created attachment 380456 [details]
Patch
Comment 7 Eric Carlson 2019-10-09 10:08:44 PDT
The mac-wk1 failure is unrelated.
Comment 8 WebKit Commit Bot 2019-10-09 11:10:17 PDT
Comment on attachment 380456 [details]
Patch

Clearing flags on attachment: 380456

Committed r250918: <https://trac.webkit.org/changeset/250918>
Comment 9 WebKit Commit Bot 2019-10-09 11:10:19 PDT
All reviewed patches have been landed.  Closing bug.