RESOLVED CONFIGURATION CHANGED257393
Speedometer's flight test has timeouts, which lead to non-determinism 
https://bugs.webkit.org/show_bug.cgi?id=257393
Summary Speedometer's flight test has timeouts, which lead to non-determinism
Scott Violet
Reported 2023-05-26 14:12:05 PDT
See https://github.com/WebKit/WebKit/blob/main/PerformanceTests/Speedometer2.1/resources/todomvc/dependency-examples/flight/flight/app/js/main.js#L27 . The test does 6 timeouts, which can lead to non-determinism. If the test needs to wait until something is loaded, that should not be included in test time.
Attachments
Add attachment proposed patch, testcase, etc.
Ryosuke Niwa
Comment 1 2023-05-27 18:52:16 PDT
Speedometer 3 benchmark, which is being developed at https://github.com/WebKit/Speedometer/pull/144 does not have this workload.
Scott Violet
Comment 2 2023-06-05 09:00:24 PDT
Isn't that ~6 months out? Seems like it would be good to fix this before the next version of Speedometer.
Ryosuke Niwa
Comment 3 2023-06-05 09:02:51 PDT
(In reply to Scott Violet from comment #2) > Isn't that ~6 months out? Seems like it would be good to fix this before the > next version of Speedometer. This workload is disabled by default anyway so it won't affect the official benchmark result.
Ryosuke Niwa
Comment 4 2023-06-05 09:03:43 PDT
(In reply to Ryosuke Niwa from comment #3) > (In reply to Scott Violet from comment #2) > > Isn't that ~6 months out? Seems like it would be good to fix this before the > > next version of Speedometer. > > This workload is disabled by default anyway so it won't affect the official > benchmark result. Actually, that's not true. This is about FlightJS's TodoMVC, which is included in the official result. Regardless, we're not going to fix this for 2.1.
Note You need to log in before you can comment on or make changes to this bug.