305926 2026-01-21 08:55:20 -0800 [WPE][ARM 32-bits]:REGRESSION(305815@main-305896@main): All subtests of JetStream2 timeout and WebProcess uses 100% of CPU. 2026-01-22 14:26:24 -0800 1 1 1 Unclassified WebKit WPE WebKit WebKit Nightly Build Unspecified Unspecified RESOLVED DUPLICATE 306062 https://bugs.webkit.org/show_bug.cgi?id=305723 https://bugs.webkit.org/show_bug.cgi?id=306062 P2 Normal --- 1 clopez webkit-unassigned angelos annevk bugs-noreply jmichaud oldest_to_newest 2173679 0 clopez 2026-01-21 08:55:20 -0800 This has been detected in the WPE RPi4 perf bots for ARM32. * Last good run at 305815@main https://build.webkit.org/#/builders/892/builds/18952 * First bad run at 305896@main https://build.webkit.org/#/builders/892/builds/18952 As you can see in the bad run the test step got stuck running JetStream2. It is taking 19 hours and counting. Previously it "only" took 9 hours to run everything. Previously only a few of the JetStream2 test were giving timeotus $ curl -s https://build.webkit.org/api/v2/logs/47895223/raw_inline | grep Timeout 2026-01-19 14:09:45,163 - ERROR - subtest OfflineAssembler of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-19 16:47:27,587 - ERROR - subtest hello_world-LJF of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-19 17:20:46,921 - ERROR - subtest list_search-LJF of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-19 17:54:06,208 - ERROR - subtest lists-LJF of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-19 18:27:25,527 - ERROR - subtest string_lists-LJF of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered But now **all** of the jetstream2 subtests are timing out $ curl -s https://build.webkit.org/api/v2/logs/47918864/raw_inline | grep Timeout 2026-01-20 13:44:17,613 - ERROR - subtest Air of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 14:17:36,926 - ERROR - subtest Basic of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 14:50:56,198 - ERROR - subtest ML of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 15:24:15,462 - ERROR - subtest Babylon of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 15:57:34,622 - ERROR - subtest cdjs of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 16:30:53,807 - ERROR - subtest first-inspector-code-load of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 17:04:13,009 - ERROR - subtest multi-inspector-code-load of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 17:37:32,276 - ERROR - subtest Box2D of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 18:10:51,491 - ERROR - subtest octane-code-load of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 18:44:10,683 - ERROR - subtest crypto of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 19:17:29,911 - ERROR - subtest delta-blue of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 19:50:49,121 - ERROR - subtest earley-boyer of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 20:24:08,363 - ERROR - subtest gbemu of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 20:57:27,531 - ERROR - subtest mandreel of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 21:30:46,756 - ERROR - subtest navier-stokes of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 22:04:05,981 - ERROR - subtest pdfjs of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 22:37:25,198 - ERROR - subtest raytrace of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 23:10:44,549 - ERROR - subtest regexp of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 23:44:03,903 - ERROR - subtest richards of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 00:17:23,152 - ERROR - subtest splay of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 00:50:42,340 - ERROR - subtest typescript of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 01:24:01,615 - ERROR - subtest octane-zlib of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 01:57:20,910 - ERROR - subtest FlightPlanner of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 02:30:40,216 - ERROR - subtest OfflineAssembler of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 03:03:59,434 - ERROR - subtest UniPoker of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 03:37:18,709 - ERROR - subtest async-fs of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 04:10:37,911 - ERROR - subtest float-mm.c of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 04:43:57,147 - ERROR - subtest hash-map of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 05:17:16,523 - ERROR - subtest ai-astar of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 05:50:35,707 - ERROR - subtest gaussian-blur of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 06:23:54,912 - ERROR - subtest stanford-crypto-aes of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 06:57:14,047 - ERROR - subtest stanford-crypto-pbkdf2 of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 07:30:33,361 - ERROR - subtest stanford-crypto-sha256 of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 08:03:52,585 - ERROR - subtest json-stringify-inspector of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 08:37:11,861 - ERROR - subtest json-parse-inspector of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered [...still suite test-run unfinished as of writing this...] I entered into the board and I can see WPEWebProcess using 100% of the CPU, 2% of the RAM (4GB total). This are the commits related to JSC in the range $ git log --oneline 3a8fc06e2e25f38b2051ca2dd6c4a78bec5756c6..0b1706ea5580e9617b324814f95a476ad2d31bf1 Source/JavaScriptCore/ d21256503ee6 [JSC] `RegExp.prototype [ %Symbol.split% ]` Incorrectly Uses Fast Path when `limit` is not a Number https://bugs.webkit.org/show_bug.cgi?id=305192 b932c71d5e15 Address some unsafe cast warning in Internals.cpp https://bugs.webkit.org/show_bug.cgi?id=305808 bbf8d344a5bb [JSC] Use `hasSpecialProperties` bit for cheking `constructor` properties https://bugs.webkit.org/show_bug.cgi?id=305801 c0deda0b0cc5 [JSC] Handle `String.fromCharCode` in abstract interpreter https://bugs.webkit.org/show_bug.cgi?id=194308 38411ab91e01 [JSC] Optimize `[...set]` https://bugs.webkit.org/show_bug.cgi?id=305446 986543b9d0e8 [JSC] Handle `StringCodePointAt` in Abastract Interpreter https://bugs.webkit.org/show_bug.cgi?id=305629 4211cd058e41 [JSC][WASM][Debugger] Fix instance registration race and use InstanceAnchor for lifecycle management rdar://168389785 https://bugs.webkit.org/show_bug.cgi?id=305714 3e93d47f3f58 Allow implicit conversion from RefPtr<T> to T* https://bugs.webkit.org/show_bug.cgi?id=305754 2173714 1 clopez 2026-01-21 10:25:26 -0800 This has been detected in the WPE RPi4 perf bots for ARM32. * Last good run at 305815@main https://build.webkit.org/#/builders/892/builds/18952 * First bad run at 305896@main https://build.webkit.org/#/builders/892/builds/18960 As you can see in the bad run the test step got stuck running JetStream2. It is taking 19 hours and counting. Previously it "only" took 9 hours to run everything. Previously only a few of the JetStream2 test were giving timeotus $ curl -s https://build.webkit.org/api/v2/logs/47895223/raw_inline | grep Timeout 2026-01-19 14:09:45,163 - ERROR - subtest OfflineAssembler of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-19 16:47:27,587 - ERROR - subtest hello_world-LJF of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-19 17:20:46,921 - ERROR - subtest list_search-LJF of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-19 17:54:06,208 - ERROR - subtest lists-LJF of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-19 18:27:25,527 - ERROR - subtest string_lists-LJF of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered But now **all** of the jetstream2 subtests are timing out $ curl -s https://build.webkit.org/api/v2/logs/47918864/raw_inline | grep Timeout 2026-01-20 13:44:17,613 - ERROR - subtest Air of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 14:17:36,926 - ERROR - subtest Basic of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 14:50:56,198 - ERROR - subtest ML of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 15:24:15,462 - ERROR - subtest Babylon of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 15:57:34,622 - ERROR - subtest cdjs of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 16:30:53,807 - ERROR - subtest first-inspector-code-load of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 17:04:13,009 - ERROR - subtest multi-inspector-code-load of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 17:37:32,276 - ERROR - subtest Box2D of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 18:10:51,491 - ERROR - subtest octane-code-load of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 18:44:10,683 - ERROR - subtest crypto of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 19:17:29,911 - ERROR - subtest delta-blue of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 19:50:49,121 - ERROR - subtest earley-boyer of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 20:24:08,363 - ERROR - subtest gbemu of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 20:57:27,531 - ERROR - subtest mandreel of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 21:30:46,756 - ERROR - subtest navier-stokes of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 22:04:05,981 - ERROR - subtest pdfjs of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 22:37:25,198 - ERROR - subtest raytrace of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 23:10:44,549 - ERROR - subtest regexp of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-20 23:44:03,903 - ERROR - subtest richards of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 00:17:23,152 - ERROR - subtest splay of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 00:50:42,340 - ERROR - subtest typescript of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 01:24:01,615 - ERROR - subtest octane-zlib of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 01:57:20,910 - ERROR - subtest FlightPlanner of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 02:30:40,216 - ERROR - subtest OfflineAssembler of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 03:03:59,434 - ERROR - subtest UniPoker of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 03:37:18,709 - ERROR - subtest async-fs of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 04:10:37,911 - ERROR - subtest float-mm.c of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 04:43:57,147 - ERROR - subtest hash-map of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 05:17:16,523 - ERROR - subtest ai-astar of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 05:50:35,707 - ERROR - subtest gaussian-blur of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 06:23:54,912 - ERROR - subtest stanford-crypto-aes of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 06:57:14,047 - ERROR - subtest stanford-crypto-pbkdf2 of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 07:30:33,361 - ERROR - subtest stanford-crypto-sha256 of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 08:03:52,585 - ERROR - subtest json-stringify-inspector of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered 2026-01-21 08:37:11,861 - ERROR - subtest json-parse-inspector of benchmark plan jetstream2 failed with exception: Timeout alarm was triggered [...still suite test-run unfinished as of writing this...] I entered into the board and I can see WPEWebProcess using 100% of the CPU, 2% of the RAM (4GB total). This are the commits related to JSC in the range $ git log --oneline 3a8fc06e2e25f38b2051ca2dd6c4a78bec5756c6..0b1706ea5580e9617b324814f95a476ad2d31bf1 Source/JavaScriptCore/ d21256503ee6 [JSC] `RegExp.prototype [ %Symbol.split% ]` Incorrectly Uses Fast Path when `limit` is not a Number https://bugs.webkit.org/show_bug.cgi?id=305192 b932c71d5e15 Address some unsafe cast warning in Internals.cpp https://bugs.webkit.org/show_bug.cgi?id=305808 bbf8d344a5bb [JSC] Use `hasSpecialProperties` bit for cheking `constructor` properties https://bugs.webkit.org/show_bug.cgi?id=305801 c0deda0b0cc5 [JSC] Handle `String.fromCharCode` in abstract interpreter https://bugs.webkit.org/show_bug.cgi?id=194308 38411ab91e01 [JSC] Optimize `[...set]` https://bugs.webkit.org/show_bug.cgi?id=305446 986543b9d0e8 [JSC] Handle `StringCodePointAt` in Abastract Interpreter https://bugs.webkit.org/show_bug.cgi?id=305629 4211cd058e41 [JSC][WASM][Debugger] Fix instance registration race and use InstanceAnchor for lifecycle management rdar://168389785 https://bugs.webkit.org/show_bug.cgi?id=305714 3e93d47f3f58 Allow implicit conversion from RefPtr<T> to T* https://bugs.webkit.org/show_bug.cgi?id=305754 2174026 2 clopez 2026-01-22 00:59:42 -0800 Angelos comments that the JSCOnly post-commit bot also appears to be extremely slow, timing out after 20h (normally takes less than half an hour) We can see there this: - First bad build at 305865@main (20 hours, timeout) https://build.webkit.org/#/builders/24/builds/60635 - Last bad build at 305856@main (19 minutes) https://build.webkit.org/#/builders/24/builds/60634 So that narrows the commit range to [305856@main-305865@main] which gives us 9 commits, but no one of them touches Source/JavaScriptCore (??) $ git log --oneline b932c71d5e15603ac2cd43c7552d918ed161a64e..54b3bb5bb0669034734b7bf49a584693165945b7 54b3bb5bb066 Facebook video call web page is promoted to now playing info application rdar://168095616 https://bugs.webkit.org/show_bug.cgi?id=305469 fbbd9a28caa2 Convert HashMap<..., RefPtr to Ref in NetworkProcess https://bugs.webkit.org/show_bug.cgi?id=305723 89ad6512372c Reduce unsafeness in WebCore/css/query https://bugs.webkit.org/show_bug.cgi?id=304399 rdar://167168414 4b75ae7073f0 [WPE] Further cleanup after removal of the Qt5 embedding API https://bugs.webkit.org/show_bug.cgi?id=305602 1d511d6cf0ce Address some unsafe cast warnings in Source/WebCore/layout https://bugs.webkit.org/show_bug.cgi?id=305810 c5f94bb4b981 Address unsafe cast issues in Source/WebCore/bindings https://bugs.webkit.org/show_bug.cgi?id=305798 d21d3038fbce WPT service-worker/tentative/static-router/static-router-resource-timing.https.html is no longer flaky in cocoa bots rdar://168444917 https://bugs.webkit.org/show_bug.cgi?id=305766 e27ac61ee273 getUserMedia with constraints where max frameRate bigger than 16 does not work rdar://115713922 https://bugs.webkit.org/show_bug.cgi?id=261461 8123c8a002e5 Address some unsafe cast warnings in Source/WebCore/svg https://bugs.webkit.org/show_bug.cgi?id=305805 I'm puzzled. 2174027 3 clopez 2026-01-22 01:06:41 -0800 Angelos comments that the JSCOnly post-commit bot also appears to be extremely slow, timing out after 20h (normally takes less than half an hour) We can see there this: - First bad build at 305865@main (20 hours, timeout) https://build.webkit.org/#/builders/24/builds/60635 - Last good build at 305856@main (19 minutes) https://build.webkit.org/#/builders/24/builds/60634 So that narrows the commit range to [305856@main-305865@main] which gives us 9 commits, but no one of them touches Source/JavaScriptCore (??) $ git log --oneline b932c71d5e15603ac2cd43c7552d918ed161a64e..54b3bb5bb0669034734b7bf49a584693165945b7 54b3bb5bb066 Facebook video call web page is promoted to now playing info application rdar://168095616 https://bugs.webkit.org/show_bug.cgi?id=305469 fbbd9a28caa2 Convert HashMap<..., RefPtr to Ref in NetworkProcess https://bugs.webkit.org/show_bug.cgi?id=305723 89ad6512372c Reduce unsafeness in WebCore/css/query https://bugs.webkit.org/show_bug.cgi?id=304399 rdar://167168414 4b75ae7073f0 [WPE] Further cleanup after removal of the Qt5 embedding API https://bugs.webkit.org/show_bug.cgi?id=305602 1d511d6cf0ce Address some unsafe cast warnings in Source/WebCore/layout https://bugs.webkit.org/show_bug.cgi?id=305810 c5f94bb4b981 Address unsafe cast issues in Source/WebCore/bindings https://bugs.webkit.org/show_bug.cgi?id=305798 d21d3038fbce WPT service-worker/tentative/static-router/static-router-resource-timing.https.html is no longer flaky in cocoa bots rdar://168444917 https://bugs.webkit.org/show_bug.cgi?id=305766 e27ac61ee273 getUserMedia with constraints where max frameRate bigger than 16 does not work rdar://115713922 https://bugs.webkit.org/show_bug.cgi?id=261461 8123c8a002e5 Address some unsafe cast warnings in Source/WebCore/svg https://bugs.webkit.org/show_bug.cgi?id=305805 I'm puzzled. 2174048 4 angelos 2026-01-22 03:16:56 -0800 After some bisecting, I can confirm that the extreme slowdown is a result of this change to `HashTraits.h`: https://github.com/WebKit/WebKit/commit/fbbd9a28caa2a79f3bcb40a184cab7c63c0e23f7#diff-6d87d8d923fe599f98c8af6f8dea0fa7f1eff31fc577cdccefa015176ccb54f2 2174250 5 clopez 2026-01-22 14:26:24 -0800 305864@main got reverted in 306040@main So I'm closing this issue. *** This bug has been marked as a duplicate of bug 306062 ***