Bug 171703 – [ Sierra Debug WK2] Layout Test webrtc/libwebrtc/descriptionGetters.html is flaky

• Component: New Bugs
• Status: NEW

[Matt Lewis]

The test webrtc/libwebrtc/descriptionGetters.html is flaky on Sierra wk2 Debug.

There was a change in changeset r215251 on the teston 4/11/2017 but the first failure was on 4/7/2017

History:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=webrtc%2Flibwebrtc%2FdescriptionGetters.html

Build log:
https://build.webkit.org/builders/Apple%20Sierra%20Debug%20WK2%20(Tests)/builds/818

Diff:
--- /Volumes/Data/slave/sierra-debug-tests-wk2/build/layout-test-results/webrtc/libwebrtc/descriptionGetters-expected.txt
+++ /Volumes/Data/slave/sierra-debug-tests-wk2/build/layout-test-results/webrtc/libwebrtc/descriptionGetters-actual.txt
@@ -1,3 +1,3 @@
 
-PASS description getters when changing description from video to video & data channel 
+FAIL description getters when changing description from video to video & data channel assert_equals: test 14 expected "v=0\r\no=- 8753924785808259018 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE video\r\na=msid-semantic: WMS b0882ffc-8fd3-4947-9cd7-4a437e3ce559\r\nm=video 61530 UDP/TLS/RTP/SAVPF 96 98 99 97 100\r\nc=IN IP4 17.202.88.219\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=candidate:998840395 1 udp 2122260223 17.202.88.219 61530 typ host generation 0 network-id 1 network-cost 50\r\na=ice-ufrag:qqD6\r\na=ice-pwd:RF2vPRr1xgERO0LnO/FPWc12\r\na=fingerprint:sha-256 0E:A9:35:59:0D:9C:3A:47:9B:8E:9C:78:31:2A:F7:A7:5F:22:28:67:16:F8:CF:80:77:0E:B8:60:18:73:C2:F9\r\na=setup:actpass\r\na=mid:video\r\na=extmap:2 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:4 urn:3gpp:video-orientation\r\na=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\na=sendrecv\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 red/90000\r\na=rtpmap:98 ulpfec/90000\r\na=rtpmap:99 H264/90000\r\na=rtcp-fb:99 ccm fir\r\na=rtcp-fb:99 nack\r\na=rtcp-fb:99 nack pli\r\na=rtcp-fb:99 goog-remb\r\na=rtcp-fb:99 transport-cc\r\na=fmtp:99 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=rtpmap:97 rtx/90000\r\na=fmtp:97 apt=96\r\na=rtpmap:100 rtx/90000\r\na=fmtp:100 apt=99\r\na=ssrc-group:FID 1624265679 1637652785\r\na=ssrc:1624265679 cname:8zp4f4S054Vz02Zj\r\na=ssrc:1624265679 msid:b0882ffc-8fd3-4947-9cd7-4a437e3ce559 3213e84d-763b-4a13-b3db-3b0e1a34537b\r\na=ssrc:1624265679 mslabel:b0882ffc-8fd3-4947-9cd7-4a437e3ce559\r\na=ssrc:1624265679 label:3213e84d-763b-4a13-b3db-3b0e1a34537b\r\na=ssrc:1637652785 cname:8zp4f4S054Vz02Zj\r\na=ssrc:1637652785 msid:b0882ffc-8fd3-4947-9cd7-4a437e3ce559 3213e84d-763b-4a13-b3db-3b0e1a34537b\r\na=ssrc:1637652785 mslabel:b0882ffc-8fd3-4947-9cd7-4a437e3ce559\r\na=ssrc:1637652785 label:3213e84d-763b-4a13-b3db-3b0e1a34537b\r\n" but got "v=0\r\no=- 8753924785808259018 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE video\r\na=msid-semantic: WMS b0882ffc-8fd3-4947-9cd7-4a437e3ce559\r\nm=video 9 UDP/TLS/RTP/SAVPF 96 98 99 97 100\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:qqD6\r\na=ice-pwd:RF2vPRr1xgERO0LnO/FPWc12\r\na=fingerprint:sha-256 0E:A9:35:59:0D:9C:3A:47:9B:8E:9C:78:31:2A:F7:A7:5F:22:28:67:16:F8:CF:80:77:0E:B8:60:18:73:C2:F9\r\na=setup:actpass\r\na=mid:video\r\na=extmap:2 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:4 urn:3gpp:video-orientation\r\na=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\na=sendrecv\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 red/90000\r\na=rtpmap:98 ulpfec/90000\r\na=rtpmap:99 H264/90000\r\na=rtcp-fb:99 ccm fir\r\na=rtcp-fb:99 nack\r\na=rtcp-fb:99 nack pli\r\na=rtcp-fb:99 goog-remb\r\na=rtcp-fb:99 transport-cc\r\na=fmtp:99 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=rtpmap:97 rtx/90000\r\na=fmtp:97 apt=96\r\na=rtpmap:100 rtx/90000\r\na=fmtp:100 apt=99\r\na=ssrc-group:FID 1624265679 1637652785\r\na=ssrc:1624265679 cname:8zp4f4S054Vz02Zj\r\na=ssrc:1624265679 msid:b0882ffc-8fd3-4947-9cd7-4a437e3ce559 3213e84d-763b-4a13-b3db-3b0e1a34537b\r\na=ssrc:1624265679 mslabel:b0882ffc-8fd3-4947-9cd7-4a437e3ce559\r\na=ssrc:1624265679 label:3213e84d-763b-4a13-b3db-3b0e1a34537b\r\na=ssrc:1637652785 cname:8zp4f4S054Vz02Zj\r\na=ssrc:1637652785 msid:b0882ffc-8fd3-4947-9cd7-4a437e3ce559 3213e84d-763b-4a13-b3db-3b0e1a34537b\r\na=ssrc:1637652785 mslabel:b0882ffc-8fd3-4947-9cd7-4a437e3ce559\r\na=ssrc:1637652785 label:3213e84d-763b-4a13-b3db-3b0e1a34537b\r\n"

[Matt Lewis]

Created attachment 309120 [details]
Mark test as flaky

[Ryan Haddad]

Comment on attachment 309120 [details]
Mark test as flaky

Clearing flags on attachment: 309120

Committed r216222: <http://trac.webkit.org/changeset/216222>

[Radar WebKit Bug Importer]

<rdar://problem/32008597>

[Matt Lewis]

This test has started to become flaky on macOS El Capitan WK2 as well:
https://build.webkit.org/results/Apple%20El%20Capitan%20Debug%20WK2%20(Tests)/r218503%20(1821)/results.html

Adjusted the test expectations:
https://trac.webkit.org/changeset/218514/webkit

[youenn fablet]

Created attachment 322208 [details]
Patch