This means we're losing a lot of coverage. Not sure who manages these bots...
This is because perf bots' cycle time is significantly longer than that of builders especially when changes are trivial. perf. bots get behind, and hits master's threshold to delete uploaded binaries. I've asked wms to increase the threadshold but it didn't help. The only thing it did was increasing the delay and backlogs.
The only solution to this problem is reducing cycle time of perf bots. Otherwise, the backlog will grow indefinitely.
Do we know which tests are the slowest? Maybe we can get a better cycle time just by fixing a dozen or so tests.
Yeah, most of them are Dromaeo tests. We spend roughly 30% of time running the slowest 5 tests. You can process the stdout on bots to find the ranking although I've posted somewhere on Bugzilla as well.
We don't have any workable solution for this at the moment.