263260 2023-10-17 10:49:54 -0700 Move testharness.js tests to using a timeout_multiplier 2023-10-17 10:50:02 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Nightly Build Unspecified Unspecified NEW https://bugs.webkit.org/show_bug.cgi?id=248694 https://bugs.webkit.org/show_bug.cgi?id=263252 InRadar P2 Normal --- 1 gsnedders webkit-unassigned webkit-bug-importer oldest_to_newest 1985606 0 gsnedders 2023-10-17 10:49:54 -0700 Currently we set the internal testharness.js timeout in our testharnessreport.js, totally overriding testharness.js's default behaviours: setTimeout(timeout, testRunner.timeout * 0.9); For Apple ports (though see also bug 263252), this gives for non-slow tests in most configurations a 27s timeout, and for slow tests in most configurations a 135s timeout. Upstream wptrunner uses a 10s timeout for most tests, or a 60s timeout for "long" (i.e., slow) tests. That said, this is configurable in a bunch of ways; debug Firefox for example uses a 3x multiplier and therefore ends up with 30s/180s timeouts. (Within WebKit, the glib port uses double the timeout for debug, which would mean 20s/120s timeouts, still both shorter than what we currently use.) In general, we should probably also be setting timeout_multiplier rather than overriding the timeout directly: this allows tests to use test.step_timeout() which scales its timeout based on the multiplier, which hopefully saves us some flakiness. There's two main approaches we could go with here: One is to set it based on the current TestRunner timeout and the default WPT timeout (something like testRunner.timeout * 0.9 / wptTimeout would maintain our current behaviour, but would always result in >1.0 multipliers, which is probably not necessary so we might want a smaller constant than 0.9). The other is to actually change how we deal with timeouts throughout run-webkit-tests (which probably isn't a bad idea in general!) to have a base timeout and then just repeatedly increase the multiplier, and pass both into the test runner. 1985607 1 webkit-bug-importer 2023-10-17 10:50:02 -0700 <rdar://problem/117083423>