Bug 298471

Summary: fast/frames/lots-of-iframes.html and fast/frames/lots-of-objects.html are flaky timeouts
Product: WebKit Reporter: Jonathan Bedard <jbedard>
Component: FramesAssignee: Ryan Reno <rreno>
Status: RESOLVED FIXED    
Severity: Normal CC: rreno, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=299184
Attachments:
Description Flags
Sonoma Debug Regression Point none

Jonathan Bedard
Reported 2025-09-05 13:23:32 PDT
After 296692@main, fast/frames/lots-of-objects.html and fast/frames/lots-of-iframes.html started consistently failing on a few wk1 configurations. History: https://results.webkit.org/?suite=layout-tests&suite=layout-tests&test=fast%2Fframes%2Flots-of-objects.html&test=fast%2Fframes%2Flots-of-iframes.html&flavor=wk1&limit=10000 I was able to trivially reproduce locally with 'run-webkit-tests --root ../buildToTest fast/frames/lots-of-objects.html fast/frames/lots-of-iframes.html' on a Debug build build. No doubt this is related to webkit.org/b/121452, I'm skeptical of the utility of these tests generally, but also, 296692@main is clearly a regression, and might generally be a performance regression for a specific type of workload.
Attachments
Sonoma Debug Regression Point (334.58 KB, image/png)
2025-09-08 13:42 PDT, Jonathan Bedard 		no flags
Details
View All Add attachment proposed patch, testcase, etc.
Radar WebKit Bug Importer
Comment 1 2025-09-05 13:23:53 PDT
<rdar://problem/159964738>
Jonathan Bedard
Comment 2 2025-09-05 14:27:59 PDT
Pull request: https://github.com/WebKit/WebKit/pull/50385
EWS
Comment 3 2025-09-05 14:30:30 PDT
Committed 299643@main (6b9dcaab6c89): <https://commits.webkit.org/299643@main> Reviewed commits have been landed. Closing PR #50385 and removing active labels.
Alexey Proskuryakov
Comment 4 2025-09-07 18:12:48 PDT
This commit only updated TestExpectations, re-opening.
Alexey Proskuryakov
Comment 5 2025-09-07 18:17:51 PDT
296692@main doesn't seem like the correct regression point, these tests were timing out before that, too
Jonathan Bedard
Comment 6 2025-09-08 13:42:06 PDT
Created attachment 476676 [details] Sonoma Debug Regression Point
Jonathan Bedard
Comment 7 2025-09-08 13:47:47 PDT
296692@main regressed something (Sonoma, in particular, regressed over this commit) It's certainly not the only regression point though.
Ryan Reno
Comment 8 2025-09-17 20:12:37 PDT
These tests have likely been flaky timeouts for over a decade. Chrome removed them back in 2013(!!) as part of a purge because the tests were flaky timeouts in their infrastructure. From the results link in the original report these tests have been timing out since Sonoma H debug had data. I don't see how it is possible that the blamed commit caused a regression here. All that said, I tried my hand at optimizing the tests and managed to cut the runtime approximately in half as measured with the following command on my M4 Max MacBookPro. ``` % time run-webkit-tests --debug --no-build --iterations 5 --no-retry-failures --exit-after-n-failures 1 -1 fast/frames/lots-of-iframes.html ``` This is what I did. Some of these technologies weren't available back when the test was written. * <!DOCTYPE html> (surprisingly seemed to cut average runtime per test iteration down by about 200ms??) * Use JavaScript APIs to create frames instead of using string concatenation and innerHTML. * Append frames to a DocumentFragment while iterating then append the whole fragment to the body at the end. * set iframe's display to 'none' * move the frames offscreen + use loading="lazy" * use srcdoc instead of src for the frame document's content. I haven't measured the lots-of-objects test yet but I suspect we can see a win using the same techniques.
Ryan Reno
Comment 9 2025-09-17 21:27:04 PDT
I was only able to achieve about a 25% improvement in runtime with an equivalent command for fast/frames/lots-of-objects.html. This is because object elements don't have lazy loading and if I use display: none they never try to load their documents and so the 1001th object is not blocked, nullifying the point of the test. I used MutationObserver instead of load events in conjunction with a DocumentFragment for appending the objects to get the speed up. Patch incoming.
Ryan Reno
Comment 10 2025-09-17 21:36:59 PDT
Pull request: https://github.com/WebKit/WebKit/pull/50916
EWS
Comment 11 2025-09-18 11:52:32 PDT
Committed 300174@main (4b9d31c3a999): <https://commits.webkit.org/300174@main> Reviewed commits have been landed. Closing PR #50916 and removing active labels.
Ryan Reno
Comment 12 2025-09-19 07:48:01 PDT
Still timing out on the bots, but fast/frames/lots-of-iframes got a lot better. Might be worth just skipping in debug, these tests are slow.
Note You need to log in before you can comment on or make changes to this bug.