177615 2017-09-28 13:08:35 -0700 REGRESSION (High Sierra, iOS 11): LayoutTest imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/successes_RSA-PSS.worker.html is very slow, and a flaky failure 2017-10-02 14:19:13 -0700 1 1 1 Unclassified WebKit New Bugs WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED InRadar P2 Normal --- 1 ryanhaddad webkit-unassigned jiewen_tan jlewis3 webkit-bug-importer youennf oldest_to_newest 1354250 0 ryanhaddad 2017-09-28 13:08:35 -0700 LayoutTest imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/successes_RSA-PSS.worker.html is a flaky failure https://build.webkit.org/results/Apple%20High%20Sierra%20Debug%20WK2%20(Tests)/r222613%20(35)/results.html https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2FWebCryptoAPI%2FgenerateKey%2Fsuccesses_RSA-PSS.worker.html --- /Volumes/Data/slave/highsierra-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/successes_RSA-PSS.worker-expected.txt +++ /Volumes/Data/slave/highsierra-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/successes_RSA-PSS.worker-actual.txt @@ -1,3 +1,5 @@ + +Harness Error (TIMEOUT), message = null PASS Success: generateKey({hash: SHA-1, modulusLength: 2048, name: RSA-PSS, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [sign]) PASS Success: generateKey({hash: SHA-1, modulusLength: 2048, name: RSA-PSS, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [sign]) @@ -25,14 +27,14 @@ PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [sign, verify, sign, sign, verify]) PASS Success: generateKey({hash: SHA-1, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [sign]) PASS Success: generateKey({hash: SHA-1, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [sign]) -PASS Success: generateKey({hash: SHA-1, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [verify, sign]) -PASS Success: generateKey({hash: SHA-1, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [verify, sign]) -PASS Success: generateKey({hash: SHA-1, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [sign, verify, sign, sign, verify]) -PASS Success: generateKey({hash: SHA-1, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [sign, verify, sign, sign, verify]) -PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [sign]) -PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [sign]) -PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [verify, sign]) -PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [verify, sign]) -PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [sign, verify, sign, sign, verify]) -PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [sign, verify, sign, sign, verify]) +TIMEOUT Success: generateKey({hash: SHA-1, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [verify, sign]) Test timed out +NOTRUN Success: generateKey({hash: SHA-1, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [verify, sign]) +NOTRUN Success: generateKey({hash: SHA-1, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [sign, verify, sign, sign, verify]) +NOTRUN Success: generateKey({hash: SHA-1, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [sign, verify, sign, sign, verify]) +NOTRUN Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [sign]) +NOTRUN Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [sign]) +NOTRUN Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [verify, sign]) +NOTRUN Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [verify, sign]) +NOTRUN Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [sign, verify, sign, sign, verify]) +NOTRUN Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [sign, verify, sign, sign, verify]) 1354252 1 ryanhaddad 2017-09-28 13:10:41 -0700 This test fails more often than it passes on High Sierra Debug WK2. 1354281 2 ryanhaddad 2017-09-28 14:12:47 -0700 Marked test as flaky in https://trac.webkit.org/changeset/222629/webkit 1355025 3 ap 2017-10-01 11:54:05 -0700 This is a High Sierra regression - both debug and release are always very slow, taking 15-30 seconds. This test takes 1-2 seconds on Sierra. iOS 11 is slow too. 1355026 4 webkit-bug-importer 2017-10-01 11:54:27 -0700 <rdar://problem/34760820> 1355027 5 ap 2017-10-01 12:00:08 -0700 Youenn, is there a way to run this test in browser? There is some WPT magic at play, as this file only contains a comment. 1355028 6 youennf 2017-10-01 12:03:36 -0700 Use open-layout-tests script passing the path to the test from LayiutTests. It will start the wpt server and open the test at the right http url. The html is generated by the server. This is a wrapper around the corresponding .worker.js file. 1355391 7 jiewen_tan 2017-10-02 12:20:47 -0700 (In reply to Alexey Proskuryakov from comment #3) > This is a High Sierra regression - both debug and release are always very > slow, taking 15-30 seconds. This test takes 1-2 seconds on Sierra. iOS 11 is > slow too. The reason why this test is faster in Sierra is that RSA-PSS is not enabled in Sierra. Hence, all the tests will hit the NotSupport exceptions and thus return early, which makes the test faster. 1355400 8 jiewen_tan 2017-10-02 12:27:05 -0700 We should mark it slow as all the other slow RSA key generation tests. 1355408 9 jiewen_tan 2017-10-02 12:41:00 -0700 Committed r222734: <http://trac.webkit.org/changeset/222734> 1355461 10 youennf 2017-10-02 13:43:33 -0700 Usually wpt tests that are slow have some metadata attached to them and the importer detects that the test is slow and mark it as such. It might be worth fixing the test and/or raising on issue on GitHub. 1355462 11 youennf 2017-10-02 13:44:02 -0700 Or maybe the importer is missing the metadata somehow. 1355490 12 jiewen_tan 2017-10-02 14:19:13 -0700 (In reply to youenn fablet from comment #10) > Usually wpt tests that are slow have some metadata attached to them and the > importer detects that the test is slow and mark it as such. > It might be worth fixing the test and/or raising on issue on GitHub. I think it will be likely these tests haven't been added with corresponding metadata.