Bug 88548 – flakiness dashboard needs to be updated to understand cascading test expectations

• Status: NEW
• Component: Tools / Tests

[Ojan Vafai]

See http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=svg%2Fas-background-image%2Fsvg-as-background-5.html.

It only shows the DEBUG = SLOW lines. It should also show the following line from the skia expectations file:
BUGCR131189 : svg/as-background-image/svg-as-background-5.html = IMAGE

The short-term, easy fix is to append the skia expectations contents to the main expectations contents before we process it in flakiness_dashboard.js. I think we should do this in order to make the current dashboard work.

In the long-term, when we add cascading, I'm not sure we want to duplicate all that code in flakiness_dashboard.js. It seems like we should store that data in the JSON somehow. Maybe the JSON could hold the bug# + last expected outcome for that builder?

[Dirk Pranke]

We do include the expected expectations for the test in the JSON ... we can certainly add the other modifiers, though.

[Ojan Vafai]

(In reply to comment #1)
> We do include the expected expectations for the test in the JSON ... we can certainly add the other modifiers, though.

This is the other json file. :) incremental_results.json. I don't think we include the information there.

That said, it's been on my TODO list forever to get rid of incremental_results.json. But still...the aggregated results would need this information in some way since that's all the flakiness dashboard loads (and I think we probably don't want to change that).

[Dirk Pranke]

Ah, I thought we had already gotten rid of incremental_results.json.

[Dirk Pranke]

Updating the subject to the now supported and more general cascading expectations :).