Bug 11959 – Improve layout test result display

• Status: NEW
• Component: Tools / Tests

The HTML output of run-webkit-tests, used to examine test failures, can be improved in several ways. I am attaching a very crude mockup of what the display of a single test failure might look like.

Created attachment 12005 [details]
Mockup (very rough draft)

Some notes on the mockup: the slider is intended to control an SVG filter applied to the difference image to enhance small differences (basically a linear transformation). In fact, I think it might be possible to not save the difference image as a file at all and generate it on the fly using SVG.

Also, since the width is 800px, there's plenty of room for a sidebar with the list of all failing tests.

Oh, and while I don't find it particularly useful, the "alternate between actual and expected images" functionality could be preserved, activated say by clicking a small icon between the actual and expected tabs.

It would also be nice to have the text diff colorized.

[David Kilzer (:ddkilzer)]

(In reply to comment #1)
> Created an attachment (id=12005) [edit]
> Mockup (very rough draft)

None of the expected, actual or diff images show up when I open the .webarchive, although I've confirmed that they're all in file.  Not sure why, though.  If I convert the .webarchive to a "web page, complete" format, it seems to work fine.

[David Kilzer (:ddkilzer)]

Created attachment 12007 [details]
Mockup Recreated

Converted Attachment 12005 [details] to a "web page, complete" format using a Perl script I've been working on, then converted it back to a .webarchive.  Works for me even after the "original" files have been removed.

[David Kilzer (:ddkilzer)]

(In reply to comment #4)
> (In reply to comment #1)
> > Created an attachment (id=12005) [edit]
> > Mockup (very rough draft)
> 
> None of the expected, actual or diff images show up when I open the
> .webarchive, although I've confirmed that they're all in file.  Not sure why,
> though.  If I convert the .webarchive to a "web page, complete" format, it
> seems to work fine.

This is probably a bug when reloading the .webarchive, although it would be useful to have the original sources to make it reproducible.

Created attachment 12008 [details]
Mockup (gzipped)

[Ahmad Saleem]

Is it something needed right now? Thanks!

[Alexey Proskuryakov]

The diff page saw some improvements since 2006, but some of these ideas still stand.

We don't have a filter to enhance small differences, and it continues to be tricky at times. And colorization would be nice too.