Bug 171742 - [iOS Simulator] Flaky failure LayoutTest/webrtc/libwebrtc/release-while-setting-local-description.html
Summary: [iOS Simulator] Flaky failure LayoutTest/webrtc/libwebrtc/release-while-setti...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Ryan Haddad
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-05-05 11:40 PDT by Matt Lewis
Modified: 2017-05-09 15:28 PDT (History)
6 users (show)

See Also:


Attachments
Patch (1.46 KB, patch)
2017-05-05 15:39 PDT, Ryan Haddad
buildbot: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from ews107 for mac-elcapitan-wk2 (1.47 MB, application/zip)
2017-05-05 16:47 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews124 for ios-simulator-wk2 (1010.97 KB, application/zip)
2017-05-05 17:05 PDT, Build Bot
no flags Details
Patch (2.29 KB, patch)
2017-05-09 14:44 PDT, Ryan Haddad
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Matt Lewis 2017-05-05 11:40:50 PDT
The test webrtc/libwebrtc/release-while-setting-local-description.html is a flaky failure.

This test has been flaky since r215916:
https://trac.webkit.org/changeset/215916/webkit

History:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=webrtc%2Flibwebrtc%2Frelease-while-setting-local-description.html

Build log:
https://build.webkit.org/builders/Apple%20iOS%2010%20Simulator%20Debug%20WK2%20(Tests)/builds/1113

Diff:
--- /Volumes/Data/slave/ios-simulator-10-debug-tests-wk2/build/layout-test-results/webrtc/libwebrtc/release-while-setting-local-description-expected.txt
+++ /Volumes/Data/slave/ios-simulator-10-debug-tests-wk2/build/layout-test-results/webrtc/libwebrtc/release-while-setting-local-description-actual.txt
@@ -1,5 +1,4 @@
 CONSOLE MESSAGE: Unhandled Promise Rejection: TypeError: Argument 1 ('candidate') to RTCPeerConnection.addIceCandidate must be an instance of RTCIceCandidate
-CONSOLE MESSAGE: Unhandled Promise Rejection: InvalidStateError (DOM Exception 11): The object is in an invalid state.
 PASS successfullyParsed is true
 
 TEST COMPLETE
Comment 1 Joseph Pecoraro 2017-05-05 12:42:58 PDT
This looks like it is our own test? I'd suggest the test handle possible promise exceptions.
Comment 2 Ryan Haddad 2017-05-05 15:39:09 PDT
Created attachment 309226 [details]
Patch
Comment 3 Build Bot 2017-05-05 16:47:44 PDT
Comment on attachment 309226 [details]
Patch

Attachment 309226 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/3681795

New failing tests:
webrtc/libwebrtc/release-while-setting-local-description.html
Comment 4 Build Bot 2017-05-05 16:47:45 PDT
Created attachment 309241 [details]
Archive of layout-test-results from ews107 for mac-elcapitan-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews107  Port: mac-elcapitan-wk2  Platform: Mac OS X 10.11.6
Comment 5 Build Bot 2017-05-05 17:05:17 PDT
Comment on attachment 309226 [details]
Patch

Attachment 309226 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/3681768

New failing tests:
webrtc/libwebrtc/release-while-setting-local-description.html
Comment 6 Build Bot 2017-05-05 17:05:18 PDT
Created attachment 309243 [details]
Archive of layout-test-results from ews124 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews124  Port: ios-simulator-wk2  Platform: Mac OS X 10.11.6
Comment 7 Ryan Haddad 2017-05-08 10:02:30 PDT
(In reply to Build Bot from comment #6)
> Created attachment 309243 [details]
> Archive of layout-test-results from ews124 for ios-simulator-wk2
> 
> The attached test failures were seen while running run-webkit-tests on the
> ios-sim-ews.
> Bot: ews124  Port: ios-simulator-wk2  Platform: Mac OS X 10.11.6

--- /Volumes/Data/EWS/WebKit/WebKitBuild/Release-iphonesimulator/layout-test-results/webrtc/libwebrtc/release-while-setting-local-description-expected.txt
+++ /Volumes/Data/EWS/WebKit/WebKitBuild/Release-iphonesimulator/layout-test-results/webrtc/libwebrtc/release-while-setting-local-description-actual.txt
@@ -1,5 +1,3 @@
-CONSOLE MESSAGE: Unhandled Promise Rejection: TypeError: Argument 1 ('candidate') to RTCPeerConnection.addIceCandidate must be an instance of RTCIceCandidate
-CONSOLE MESSAGE: Unhandled Promise Rejection: InvalidStateError (DOM Exception 11): The object is in an invalid state.
 PASS successfullyParsed is true
 
 TEST COMPLETE
Comment 8 Ryan Haddad 2017-05-08 16:58:07 PDT
Comment 7 is the result of me trying to silence the rejection messages, so it seems that I either did the wrong thing or I need to rebaseline the test, too.
Comment 9 Ryan Haddad 2017-05-09 14:44:34 PDT
Created attachment 309544 [details]
Patch
Comment 10 WebKit Commit Bot 2017-05-09 15:28:40 PDT
Comment on attachment 309544 [details]
Patch

Clearing flags on attachment: 309544

Committed r216548: <http://trac.webkit.org/changeset/216548>
Comment 11 WebKit Commit Bot 2017-05-09 15:28:41 PDT
All reviewed patches have been landed.  Closing bug.