171605 2017-05-03 10:20:11 -0700 Multiple LayoutTest/imported/w3c/web-platform-tests/webrtc/ are flaky failures 2017-05-22 16:41:43 -0700 1 1 1 Unclassified WebKit New Bugs WebKit Nightly Build Unspecified Unspecified NEW https://bugs.webkit.org/show_bug.cgi?id=168736 https://bugs.webkit.org/show_bug.cgi?id=170819 InRadar P2 Normal --- 1 jlewis3 jlewis3 ryanhaddad webkit-bug-importer youennf oldest_to_newest 1304099 0 jlewis3 2017-05-03 10:20:11 -0700 The Test imported/w3c/web-platform-tests/webrtc/no-media-call.html is a flaky failure on Mac-wk2 The test consistently fails at the same time as: imported/w3c/web-platform-tests/webrtc/promises-call.html imported/w3c/web-platform-tests/webrtc/datachannel-emptystring.html History: https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fwebrtc%2Fno-media-call.html Build log: https://build.webkit.org/results/Apple%20Sierra%20Release%20WK2%20(Tests)/r216117%20(1135)/results.html 1304102 1 ryanhaddad 2017-05-03 10:23:26 -0700 *** Bug 170820 has been marked as a duplicate of this bug. *** 1304105 2 308922 jlewis3 2017-05-03 10:38:36 -0700 Created attachment 308922 Mark test as flaky. 1304113 3 308922 ryanhaddad 2017-05-03 10:51:22 -0700 Comment on attachment 308922 Mark test as flaky. Clearing flags on attachment: 308922 Committed r216125: <http://trac.webkit.org/changeset/216125> 1304416 4 webkit-bug-importer 2017-05-03 17:22:05 -0700 <rdar://problem/31977318> 1311571 5 jlewis3 2017-05-22 14:23:00 -0700 As of 5/22 and revision https://trac.webkit.org/changeset/217225/webkit the tests above are now failing on iOS Simulator release and Sierra release consistently. Diff for imported/w3c/web-platform-tests/webrtc/promises-call.html: --- /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/promises-call-expected.txt +++ /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/promises-call-actual.txt @@ -1,6 +1,8 @@ -PASS Can set up a basic WebRTC call with only data using promises. +Harness Error (TIMEOUT), message = null + +TIMEOUT Can set up a basic WebRTC call with only data using promises. Test timed out iceConnectionState info -First: completed -Second: connected +First: checking +Second: new Diff for imported/w3c/web-platform-tests/webrtc/datachannel-emptystring.html: --- /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/datachannel-emptystring-expected.txt +++ /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/datachannel-emptystring-actual.txt @@ -1,8 +1,7 @@ -FAIL Can send empty strings across a WebRTC data channel. assert_equals: Empty message not received expected "" but got "done" +Harness Error (TIMEOUT), message = null + +TIMEOUT Can send empty strings across a WebRTC data channel. Test timed out Messages exchanged -Sent: [empty string] -Sent: "done" -Received: "done" 1311665 6 ryanhaddad 2017-05-22 16:41:43 -0700 (In reply to Matt Lewis from comment #5) > As of 5/22 and revision https://trac.webkit.org/changeset/217225/webkit > > the tests above are now failing on iOS Simulator release and Sierra release > consistently. > > Diff for imported/w3c/web-platform-tests/webrtc/promises-call.html: > > --- > /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test- > results/imported/w3c/web-platform-tests/webrtc/promises-call-expected.txt > +++ > /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test- > results/imported/w3c/web-platform-tests/webrtc/promises-call-actual.txt > @@ -1,6 +1,8 @@ > > -PASS Can set up a basic WebRTC call with only data using promises. > +Harness Error (TIMEOUT), message = null > + > +TIMEOUT Can set up a basic WebRTC call with only data using promises. Test > timed out > iceConnectionState info > > -First: completed > -Second: connected > +First: checking > +Second: new > > Diff for imported/w3c/web-platform-tests/webrtc/datachannel-emptystring.html: > > --- > /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test- > results/imported/w3c/web-platform-tests/webrtc/datachannel-emptystring- > expected.txt > +++ > /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test- > results/imported/w3c/web-platform-tests/webrtc/datachannel-emptystring- > actual.txt > @@ -1,8 +1,7 @@ > > -FAIL Can send empty strings across a WebRTC data channel. assert_equals: > Empty message not received expected "" but got "done" > +Harness Error (TIMEOUT), message = null > + > +TIMEOUT Can send empty strings across a WebRTC data channel. Test timed out > Messages exchanged > > -Sent: [empty string] > -Sent: "done" > -Received: "done" Added iOS TestExpectations in https://trac.webkit.org/changeset/217257/webkit. 308922 2017-05-03 10:38:36 -0700 2017-05-03 10:51:50 -0700 Mark test as flaky. bug-171605-20170503103835.patch text/plain 1577 jlewis3 U3VidmVyc2lvbiBSZXZpc2lvbjogMjE2MTIzCmRpZmYgLS1naXQgYS9MYXlvdXRUZXN0cy9DaGFu Z2VMb2cgYi9MYXlvdXRUZXN0cy9DaGFuZ2VMb2cKaW5kZXggOWY5ZjE5ZmE1ZGRjN2FlMmM5NjI1 M2IyNmM5NzJhZWQxOGFiOTg0OC4uYjkxMGUyN2IwZDEyOWMzYzEzYTAzZWM1NTg5YzA3YjUwNTdj MmUwZSAxMDA2NDQKLS0tIGEvTGF5b3V0VGVzdHMvQ2hhbmdlTG9nCisrKyBiL0xheW91dFRlc3Rz L0NoYW5nZUxvZwpAQCAtMSwzICsxLDEyIEBACisyMDE3LTA1LTAzICBNYXR0IExld2lzICA8amxl d2lzM0BhcHBsZS5jb20+CisKKyAgICAgICAgTWFya2luZyB0aHJlZSBpbXBvcnRlZC93M2Mvd2Vi LXBsYXRmb3JtLXRlc3RzL3dlYnJ0YyB0ZXN0IGFzIGZsYWt5IGZhaWx1cmVzLgorICAgICAgICBo dHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9MTcxNjA1CisKKyAgICAgICAg VW5yZXZpZXdlZCB0ZXN0IGdhcmRlbmluZy4KKworICAgICAgICAqIHBsYXRmb3JtL21hYy13azIv VGVzdEV4cGVjdGF0aW9uczoKKwogMjAxNy0wNS0wMyAgSm9hbm1hcmllIERpZ2dzICA8amRpZ2dz QGlnYWxpYS5jb20+CiAKICAgICAgICAgQVg6IFRyZWF0IGNlbGxzIHdpdGggQVJJQSB0YWJsZSBj ZWxsIHByb3BlcnRpZXMgYXMgY2VsbHMKZGlmZiAtLWdpdCBhL0xheW91dFRlc3RzL3BsYXRmb3Jt L21hYy13azIvVGVzdEV4cGVjdGF0aW9ucyBiL0xheW91dFRlc3RzL3BsYXRmb3JtL21hYy13azIv VGVzdEV4cGVjdGF0aW9ucwppbmRleCBmMmQ5ZDhiYzIxOWJkMzM5MThjNWNlMTFjZDNjNDcxZTI2 ZjgzMDViLi42MWI0NjcyYzBiZDIyNjM0YzFkODRjMmQ3OTFiNjI3NGNhM2NkNmI4IDEwMDY0NAot LS0gYS9MYXlvdXRUZXN0cy9wbGF0Zm9ybS9tYWMtd2syL1Rlc3RFeHBlY3RhdGlvbnMKKysrIGIv TGF5b3V0VGVzdHMvcGxhdGZvcm0vbWFjLXdrMi9UZXN0RXhwZWN0YXRpb25zCkBAIC02NTksMyAr NjU5LDkgQEAgd2Via2l0Lm9yZy9iLzE3MDQ4NCBzd2lwZS9tYWluLWZyYW1lLXBpbm5pbmctcmVx dWlyZW1lbnQuaHRtbCBbIFBhc3MgRmFpbHVyZSBdCiB3ZWJraXQub3JnL2IvMTcwOTYwIFsgU2ll cnJhIF0gZmFzdC9tZWRpYXN0cmVhbS9nZXRVc2VyTWVkaWEtd2ViYXVkaW8uaHRtbCBbIFBhc3Mg VGltZW91dCBdCiAKIHdlYmtpdC5vcmcvYi8xNzA4MTggaW1wb3J0ZWQvdzNjL3dlYi1wbGF0Zm9y bS10ZXN0cy93ZWJydGMvcnRjcGVlcmNvbm5lY3Rpb24vc2V0UmVtb3RlRGVzY3JpcHRpb24uaHRt bCBbIFBhc3MgRmFpbHVyZSBdCisKK3dlYmtpdC5vcmcvYi8xNzE2MDUgaW1wb3J0ZWQvdzNjL3dl Yi1wbGF0Zm9ybS10ZXN0cy93ZWJydGMvbm8tbWVkaWEtY2FsbC5odG1sIFsgUGFzcyBGYWlsdXJl IF0KKword2Via2l0Lm9yZy9iLzE3MTYwNSBpbXBvcnRlZC93M2Mvd2ViLXBsYXRmb3JtLXRlc3Rz L3dlYnJ0Yy9wcm9taXNlcy1jYWxsLmh0bWwgWyBQYXNzIEZhaWx1cmUgXQorCit3ZWJraXQub3Jn L2IvMTcxNjA1IGltcG9ydGVkL3czYy93ZWItcGxhdGZvcm0tdGVzdHMvd2VicnRjL2RhdGFjaGFu bmVsLWVtcHR5c3RyaW5nLmh0bWwgWyBQYXNzIEZhaWx1cmUgXQo=