217966 2020-10-20 09:53:20 -0700 [GTK] inspector/console/queryHolders.html is failing 2022-01-05 16:12:27 -0800 1 1 1 Unclassified WebKit Web Inspector WebKit Nightly Build Unspecified Unspecified REOPENED P2 Normal --- 1 dpino webkit-unassigned bburg cgarcia inspector-bugzilla-changes oldest_to_newest 1699790 0 dpino 2020-10-20 09:53:20 -0700 The test is very flaky since r267815. Inspecting the range r267800-r267815, r267807 introduced several changes in inspector code touching WebKitGTK code. I haven't bisected the range though (before r267815 the test sporadically timeout). To reproduce the failure: $ Tools/Scripts/run-webkit-tests --gtk --release --iterations 5 inspector/console/queryHolders.html Diff: https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/r268725%20(16529)/inspector/console/queryHolders-diff.txt --- /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/inspector/console/queryHolders-expected.txt +++ /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/inspector/console/queryHolders-actual.txt @@ -75,7 +75,9 @@ [RepeatedReferenceHolder, Root] (2) -- Running test case: CommandLineAPI.queryHolders.OnlyHeldByDebugger -PASS: The result should have 0 items. +FAIL: The result should have 0 items. + Expected: 0 + Actual: 2 -- Running test case: CommandLineAPI.queryHolders.NonObject PASS: Should produce an exception. 1699920 1 dpino 2020-10-20 14:21:48 -0700 I bisected the range [r267800-r267815] and I got r267814 as the first revision where I can reproduce the regression. 1700243 2 bburg 2020-10-21 09:48:28 -0700 Carlos, I don't see anything obviously wrong which would cause this. Can you take a peek? 1700340 3 dpino 2020-10-21 13:16:15 -0700 (In reply to Brian Burg from comment #2) > Carlos, I don't see anything obviously wrong which would cause this. Can you > take a peek? It seems this same issue was tracked in bug https://bugs.webkit.org/show_bug.cgi?id=217193 Recently the implementation of `item` introduced in r267814 was renamed to `at` in r268760, among other changes. Since then the test is no longer flaky, although it sporadically times out as it used to happen before. https://results.webkit.org/?suite=layout-tests&test=inspector%2Fconsole%2FqueryHolders.html&platform=GTK&platform=WPE 1700341 4 dpino 2020-10-21 13:17:28 -0700 *** This bug has been marked as a duplicate of bug 217193 *** 1768072 5 dpino 2021-06-09 00:04:04 -0700 This test is no longer flaky in WebKitGTK as the results in the WebKitGTK post-commit bot report this test has been consistently failing in the last 4000 revisions. The current diff is: https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/r278645%20%282223%29/inspector/console/queryHolders-diff.txt --- /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/inspector/console/queryHolders-expected.txt +++ /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/inspector/console/queryHolders-actual.txt @@ -54,13 +54,13 @@ [Root] (1) -- Running test case: CommandLineAPI.queryHolders.PromiseThen -[Root, object] (2) +[Object, Root] (2) -- Running test case: CommandLineAPI.queryHolders.PromiseCatch -[Root, object] (2) +[Object, Root] (2) -- Running test case: CommandLineAPI.queryHolders.PromiseFinally -[Root, object] (2) +[Object, Root] (2) -- Running test case: CommandLineAPI.queryHolders.EventListenerCallback [<div id="event-listener-target">, Root] (2) 1828445 6 dpino 2022-01-05 16:12:27 -0800 Now this test is failing with a different output: --- /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/inspector/console/queryHolders-expected.txt +++ /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/inspector/console/queryHolders-actual.txt @@ -54,13 +54,13 @@ [Root] (1) -- Running test case: CommandLineAPI.queryHolders.PromiseThen -[Root, object] (2) +[Object, Root] (2) -- Running test case: CommandLineAPI.queryHolders.PromiseCatch -[Root, object] (2) +[Object, Root] (2) -- Running test case: CommandLineAPI.queryHolders.PromiseFinally -[Root, object] (2) +[Object, Root] (2) -- Running test case: CommandLineAPI.queryHolders.EventListenerCallback [<div id="event-listener-target">, Root] (2) @@ -75,7 +75,9 @@ [RepeatedReferenceHolder, Root] (2) -- Running test case: CommandLineAPI.queryHolders.OnlyHeldByDebugger -PASS: The result should have 0 items. +FAIL: The result should have 0 items. + Expected: 0 + Actual: 2 -- Running test case: CommandLineAPI.queryHolders.NonObject PASS: Should produce an exception. The reason for the [Root, object] and [Object, Root] difference is due to using a different label for Objects in GLib abd other WebKit ports. https://webkit-search.igalia.com/webkit/source/LayoutTests/inspector/console/queryHolders.html#83 It can be easily solved by capitalizing the propertiy names before sorting. As for the other part, the error is happening in this line: https://webkit-search.igalia.com/webkit/source/LayoutTests/inspector/console/queryHolders.html#162 I have no idea why size is 2 for GLib ports but 0 in other ports.