Bug 187408 – Layout Test imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/successes_RSA-PSS.https.any.worker.html is flaky

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Dawei Fenton (:realdawei)]

The following layout test is flaky on High Sierra Debug WK1 and Wk2

imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/successes_RSA-PSS.https.any.worker.html

Probable cause:

Unknown, researching it. 

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2FWebCryptoAPI%2FgenerateKey%2Fsuccesses_RSA-PSS.https.any.worker.html

Sample Diff:
--- /Volumes/Data/slave/highsierra-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/successes_RSA-PSS.https.any.worker-expected.txt
+++ /Volumes/Data/slave/highsierra-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/successes_RSA-PSS.https.any.worker-actual.txt
@@ -1,3 +1,5 @@
+
+Harness Error (TIMEOUT), message = null
 
 PASS Success: generateKey({hash: SHA-1, modulusLength: 2048, name: RSA-PSS, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [sign]) 
 PASS Success: generateKey({hash: SHA-1, modulusLength: 2048, name: RSA-PSS, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [sign]) 
@@ -32,7 +34,7 @@
 PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [sign]) 
 PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [sign]) 
 PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [verify, sign]) 
-PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [verify, sign]) 
-PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [sign, verify, sign, sign, verify]) 
-PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [sign, verify, sign, sign, verify]) 
+TIMEOUT Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [verify, sign]) Test timed out
+NOTRUN Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [sign, verify, sign, sign, verify]) 
+NOTRUN Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [sign, verify, sign, sign, verify])

[Dawei Fenton (:realdawei)]

This test on average runs between 20-26 seconds which puts it on the borderline -- external conditions in the test runner likely cause it to go over from time to time. 

For this reason I marked as "Slow"

https://trac.webkit.org/changeset/233591/webkit

[Jiewen Tan]

Test Expectations are updated.

[Radar WebKit Bug Importer]

<rdar://problem/42178600>