Bug 88548
| Summary: | flakiness dashboard needs to be updated to understand cascading test expectations | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Ojan Vafai <ojan> |
| Component: | Tools / Tests | Assignee: | Nobody <webkit-unassigned> |
| Status: | NEW | ||
| Severity: | Normal | CC: | dpranke, epoger |
| Priority: | P2 | ||
| Version: | 528+ (Nightly build) | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
Ojan Vafai
See http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=svg%2Fas-background-image%2Fsvg-as-background-5.html.
It only shows the DEBUG = SLOW lines. It should also show the following line from the skia expectations file:
BUGCR131189 : svg/as-background-image/svg-as-background-5.html = IMAGE
The short-term, easy fix is to append the skia expectations contents to the main expectations contents before we process it in flakiness_dashboard.js. I think we should do this in order to make the current dashboard work.
In the long-term, when we add cascading, I'm not sure we want to duplicate all that code in flakiness_dashboard.js. It seems like we should store that data in the JSON somehow. Maybe the JSON could hold the bug# + last expected outcome for that builder?
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Dirk Pranke
We do include the expected expectations for the test in the JSON ... we can certainly add the other modifiers, though.
Ojan Vafai
(In reply to comment #1)
> We do include the expected expectations for the test in the JSON ... we can certainly add the other modifiers, though.
This is the other json file. :) incremental_results.json. I don't think we include the information there.
That said, it's been on my TODO list forever to get rid of incremental_results.json. But still...the aggregated results would need this information in some way since that's all the flakiness dashboard loads (and I think we probably don't want to change that).
Dirk Pranke
Ah, I thought we had already gotten rid of incremental_results.json.
Dirk Pranke
Updating the subject to the now supported and more general cascading expectations :).