Bug 198810 - Layout Test http/wpt/webauthn/public-key-credential-create-failure-u2f-silent.https.html is flaky
Summary: Layout Test http/wpt/webauthn/public-key-credential-create-failure-u2f-silent...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Jiewen Tan
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2019-06-12 15:07 PDT by Russell Epstein
Modified: 2019-09-09 12:16 PDT (History)
5 users (show)

See Also:


Attachments
Patch (2.00 KB, patch)
2019-09-06 20:40 PDT, Jiewen Tan
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Russell Epstein 2019-06-12 15:07:27 PDT
The following layout test is flaky on Mac Debug WK2

http/wpt/webauthn/public-key-credential-create-failure-u2f-silent.https.html

Probable cause:

I believe this test has been failing since it was introduced.

I reproduced this by:

run-webkit-tests --iter 1000 http/wpt/webauthn/public-key-credential-create-failure-u2f-silent.https.html -f

Using spade 245262, where the test was introduced.

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Fwpt%2Fwebauthn%2Fpublic-key-credential-create-failure-u2f-silent.https.html

Diff:

--- /Volumes/Data/slave/mojave-debug-tests-wk2/build/layout-test-results/http/wpt/webauthn/public-key-credential-create-failure-u2f-silent.https-expected.txt
+++ /Volumes/Data/slave/mojave-debug-tests-wk2/build/layout-test-results/http/wpt/webauthn/public-key-credential-create-failure-u2f-silent.https-actual.txt
@@ -3,7 +3,7 @@
 PASS PublicKeyCredential's [[create]] with malformed U2F register response in a mock hid authenticator. 
 PASS PublicKeyCredential's [[create]] with register command error in a mock hid authenticator. 
 PASS PublicKeyCredential's [[create]] with bogus command error in a mock hid authenticator. 
-PASS PublicKeyCredential's [[create]] with first exclude credential matched in a mock hid authenticator. 
+FAIL PublicKeyCredential's [[create]] with first exclude credential matched in a mock hid authenticator. assert_throws: At least one credential matches an entry of the excludeCredentials list in the authenticator. function "function () { throw e }" threw object "NotAllowedError: Operation timed out." that is not a DOMException InvalidStateError: property "code" is equal to 0, expected 11
 PASS PublicKeyCredential's [[create]] with second exclude credential matched in a mock hid authenticator. 
 PASS PublicKeyCredential's [[create]] with first exclude credential matched in a mock hid authenticator. Test of user presence.
Comment 1 Radar WebKit Bug Importer 2019-06-12 15:09:38 PDT
<rdar://problem/51684767>
Comment 2 Jiewen Tan 2019-09-06 20:40:14 PDT
Created attachment 378270 [details]
Patch
Comment 3 dewei_zhu 2019-09-07 18:21:17 PDT
rs=me
Comment 4 Jiewen Tan 2019-09-09 10:44:10 PDT
Comment on attachment 378270 [details]
Patch

Thanks, Dewei.
Comment 5 WebKit Commit Bot 2019-09-09 12:16:22 PDT
Comment on attachment 378270 [details]
Patch

Clearing flags on attachment: 378270

Committed r249656: <https://trac.webkit.org/changeset/249656>
Comment 6 WebKit Commit Bot 2019-09-09 12:16:24 PDT
All reviewed patches have been landed.  Closing bug.