Bug 122180 – Please display information about pending runs in build.webkit.org/dashboard

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Alexey Proskuryakov]

It would be helpful to have a popover with pending run information. E.g. if I'm seeing failing tests in the last run, I'd like to quickly look up whether there is a fix already in the queue.

Not quite sure what to display - maybe a ChangeLog trimmed up to the first line with an asterisk?

[Alexey Proskuryakov]

(well, that would be a concatenation of ChangeLogs, as there are often multiple in a run).

[Timothy Hatcher]

I think the JSON contains this — an array of the changes in the run.

[Alexey Proskuryakov]

It would also help to have a link to current build buildbot page. Sometimes I wonder why things are taking too long, and it could be that bots are unhealthy.

[Tim Horton]

(In reply to comment #0)
> It would be helpful to have a popover with pending run information. E.g. if I'm seeing failing tests in the last run, I'd like to quickly look up whether there is a fix already in the queue.

+1, this would be fantastically useful

[Peter Molnar]

I started to work on this, but the term "pending run" is quite confusing, it seems this term has different meaning on this dashboard page and the Buildbot.

As far as I know, Buildbot has "pending build request", which is a revision that needs to be processed, but it's not started yet. It also has "current build", that is an actually started, but still running build. Most builders could only have 0 or 1 running builds, those who has multiple buildslaves, can have more running builds.

The dashboard displays "pending build" or "pending test run", but it's in fact not Buildbot's "pending build request", but it's the number of currently running builds/tests. 

Which one would you like to see detailed on this Dashboard page?
I assume you'd like to see both lists (running and waiting jobs), as it would make more sense (example scenario: builder tree is red, 2 patches are currently running, but there's 3 more waiting for processing, and one of them is a buildfix for the current build breakage).

So, what should we do?

Option A: to have two separate lists, one for waiting tasks ("pending build request" as Buildbot calls it), and a second list for currently running builds.

Option B: to merge those lists together some way, and also sum up the numbers of the waiting and running builds.

[Alexey Proskuryakov]

I previously posted some comments about this to bug 122191.

The question to ask here is what information a person using this tools needs to have immediately available. We should provide it, but at the same time it's critically important to keep the view uncluttered, keeping the tool immediately useful without any training.

I think that there are two aspects:

- When there are failing tests, it's very useful to know whether a fix has already been committed. So, there should be a quick way to learn about all commits made after the latest test run (more precisely, a run with the highest revision number). And that's not even limited to commits the buildbot already knows about.

A popover with appropriately shortened ChangeLogs will hopefully work for this purpose.

- When bots are not feeling well, and not processing builds/tests quickly, bot watchers should know about that too. Perhaps there could be an indication of how long the latest run is taking, which could even turn red if that's way more than previous runs. Or maybe it should turn red when there is a backlog of revisions to build/test, but the bot is idle.

Buildbot's ideas about pending runs are not particularly relevant for either aspect.

[Peter Molnar]

Created attachment 216653 [details]
Proposed patch

> - When there are failing tests, it's very useful to know whether a fix has already been committed. So, there should be a quick way to learn about all commits made after the latest test run (more precisely, a > run with the highest revision number). And that's not even limited to commits the buildbot already knows about.
> 
> A popover with appropriately shortened ChangeLogs will hopefully work for this purpose.

Attached a patch, what do you think about this kind of display? Is this suitable for your needs?

> - When bots are not feeling well, and not processing builds/tests quickly, bot watchers should know about that too. Perhaps there could be an indication of how long the latest run is taking, which could even > turn red if that's way more than previous runs. Or maybe it should turn red when there is a backlog of revisions to build/test, but the bot is idle.

I think this should be done in a separate bug, I'm planning to do this soon.

[Alexey Proskuryakov]

I applied the patch locally, and couldn't get it to work. Hovering "(2) pending test runs" doesn't show any pop-up.

A couple comments though:

1. We should get information about newer revisions from trac, not from buildbot's builder. Even if buildbot hasn't built a binary for the revision yet, it's still relevant for the purpose of knowing what else landed after the latest test run.

2. It would be better to use a real pop-up, not a title. One may want to select and copy some text, there could be rich styling, and possibly even interactive elements in the future. Tim previously suggested reusing Web Inspector code.

[Peter Molnar]

(In reply to comment #8)
> I applied the patch locally, and couldn't get it to work. Hovering "(2) pending test runs" doesn't show any pop-up.

The hover is on the revision link of the finished test runs. It displays the revision range between that test run, and the latest revision the builder of the same platform knows about. 

> A couple comments though:
> 
> 1. We should get information about newer revisions from trac, not from buildbot's builder. Even if buildbot hasn't built a binary for the revision yet, it's still relevant for the purpose of knowing what else landed after the latest test run.

Is there a significant delay here, that we have to deal with?


> 2. It would be better to use a real pop-up, not a title. One may want to select and copy some text, there could be rich styling, and possibly even interactive elements in the future. Tim previously suggested reusing Web Inspector code.

I'll check Web Inspector code.

[Alexey Proskuryakov]

> The hover is on the revision link of the finished test runs. 

This works indeed. I don't think that this is a place where people would look for it, I certainly couldn't find it myself.

I think that in addition to bug title, displaying author name/nick would be useful, if there is enough space for that.

> Is there a significant delay here, that we have to deal with?

Yes - it's been many times that I yelled at people about breakage that they fixed 5 minutes ago.

[Timothy Hatcher]

The Inspector Popover code is:

http://trac.webkit.org/browser/trunk/Source/WebInspectorUI/UserInterface/Popover.js
http://trac.webkit.org/browser/trunk/Source/WebInspectorUI/UserInterface/Popover.css

And likely some other helper types.

[Peter Gal]

Comment on attachment 216653 [details]
Proposed patch

View in context: https://bugs.webkit.org/attachment.cgi?id=216653&action=review

> Tools/BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/BuildbotQueueView.js:83
> +                linkElement.title= title;

Missing whitespace before the '=' sign.

[Alexey Proskuryakov]

Comment on attachment 216653 [details]
Proposed patch

I'm adding code to load SVN revisions from Trac in bug 122191, so we should use this data for display.

[Alexey Proskuryakov]

Created attachment 220081 [details]
proposed patch

[Alexey Proskuryakov]

Created attachment 220082 [details]
screenshot

[Alexey Proskuryakov]

Comment on attachment 220081 [details]
proposed patch

View in context: https://bugs.webkit.org/attachment.cgi?id=220081&action=review

> Tools/ChangeLog:60
> +        Like JS counterpart, mostly lifted from Web Inspector.

Oh, also added z-index:1000, because Dashboard has an element with non-zero z-index.

[Timothy Hatcher]

Comment on attachment 220081 [details]
proposed patch

Looks good to me. Nice work!

[Alexey Proskuryakov]

Committed <http://trac.webkit.org/r161120>.