217193 2020-10-01 15:25:48 -0700 REGRESSION (r267814): [ Mojave+ release wk2 ] inspector/console/queryHolders.html is a flaky failure 2020-10-21 13:21:45 -0700 1 1 1 Unclassified WebKit Web Inspector WebKit Nightly Build Unspecified Unspecified NEW https://bugs.webkit.org/show_bug.cgi?id=217115 InRadar P2 Normal --- 1 rackler webkit-unassigned aakash_jain ashvayka dpino fpizlo hi inspector-bugzilla-changes joepeck ross.kirsling webkit-bot-watchers-bugzilla webkit-bug-importer ysuzuki oldest_to_newest 1693899 0 rackler 2020-10-01 15:25:48 -0700 Description: inspector/console/queryHolders.html The first failure that I see this on the dashboard was 09/30/20 at r267815, but does not appear to be relevant to what would have caused this. I can reproduce this on r267814, but unable to reproduce on 267813 or earlier. Commit r267814 has to do with implementing an {Array, %TypedArray%, String}.prototype.item, which could have introduced the flaky failure. The change was introduced here https://trac.webkit.org/changeset/267814/webkit. REPRODUCTION STEPS Command: run-webkit-tests --exit-after-n-failures 1 --iterations 50 inspector/console/queryHolders.html Result: Regressions: Unexpected text-only failures (1) inspector/console/queryHolders.html [ Failure ] History: https://results.webkit.org/?suite=layout-tests&test=inspector%2Fconsole%2FqueryHolders.html&platform=mac Diff Log: --- /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/inspector/formatting/formatting-css-expected.txt +++ /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/inspector/formatting/formatting-css-actual.txt @@ -1,14 +1,5 @@ -Test CSS formatting. +#PID UNRESPONSIVE - WebKitTestRunner (pid 84131) +FAIL: Timed out waiting for notifyDone to be called - -== Running test suite: CSSFormatter --- Running test case: CSSFormatter -PASS: basic.css -PASS: comment.css -PASS: gradient.css -PASS: keyframes.css -PASS: media-query.css -PASS: selectors.css -PASS: url.css -PASS: wrapping.css - +#EOF +#EOF 1693900 1 webkit-bug-importer 2020-10-01 15:26:09 -0700 <rdar://problem/69858825> 1693905 2 aakash_jain 2020-10-01 15:37:26 -0700 Interesting that in https://bugs.webkit.org/show_bug.cgi?id=217115#c5 EWS blamed the patch for r267814 for this test failure. 1693908 3 rackler 2020-10-01 15:41:58 -0700 I have marked this test as failing while this issue is investigated. https://trac.webkit.org/changeset/267856/webkit 1693922 4 ross.kirsling 2020-10-01 16:30:30 -0700 The test has no overt connection to r267814, but there must be some sort of odd dependence, since it evidently wasn't flaky before. I'll try to see what possible interaction there could be... 1693977 5 ross.kirsling 2020-10-01 20:37:24 -0700 A few more details: - Repro rate seems to be around 15%. - When it fails, the queryHolders result (which is expected to be empty) contains two WI.RemoteObjects, which appear to differ only in `objectId.id`. In particular, they have the same `target` and a `description` of "Window". This very much feels like an unreliable test that just happened to be stable until something shifted underneath it...? 1693990 6 ross.kirsling 2020-10-01 23:16:02 -0700 Playing around more, it appears that adding *any* new built-in function to Array.prototype (regardless of name, order, or behavior) will cause this test to turn flaky. And yet the procedure for, say, r228266 was the same as my procedure for r267814, and this had no associated regression. I'm happy to update the test to simply expect the array to be of length 2, but I'm not sure this preserves the purpose of the test, so I'd feel more comfortable leaving it to the WI folks to adjust more suitably. 1700342 7 dpino 2020-10-21 13:17:28 -0700 *** Bug 217966 has been marked as a duplicate of this bug. *** 1700345 8 dpino 2020-10-21 13:21:45 -0700 This regression could also be reproduced in WebKitGTK. In the case of WebKitGTK, after r268760 this test is no longer flaky, although it times out sometimes as it used to happen before r267814. https://results.webkit.org/?suite=layout-tests&test=inspector%2Fconsole%2FqueryHolders.html&platform=GTK&platform=WPE