Bug 56071 – [NRWT] We should write an image to the LayoutTest result directory even if driver_output.image is empty.

• Status: RESOLVED
• Resolution: WONTFIX
• Component: Tools / Tests

[Hayato Ito]

I've found that the previous test's result image is wrongly shown in the results.html in the following scenario.

Suppose we have a test, foo.html. 

In the first run, the test run successfully and the driver_output.image was written to the LayoutTest result directory as a 'foo-actual.png'.
In the second run, suppose timeout happend. In this case, the driver_output.image was '' (empty string). So test_result_writer will skip writing the image.
As a result, the previous 'foo-actual.png' is used in results.html. That is confusing. The previoius result image was overwritten with empty_image.

[Hayato Ito]

Created attachment 85283 [details]
write-empty-image

[Hayato Ito]

I think when CRASH or TIMEOUT occurs, we don't have to use a result image.
But for some (historical?) reasons, new-run-webkit-tests tries to compare_image even in TIMEOUT occurs.
So we need this change to avoid confusion.

[Hayato Ito]

I've found that we don't need this patch for now.
When timeout occurs, the driver returns None as image_hash.
So _compare_image will exit early in this case. So results.html won't contain the link to the previous result image.

[Dirk Pranke]

presumably this should be WONTFIX or WORKSFORME instead of FIXED?

[Hayato Ito]

Yeah, I changed it to WONTFIX. Thank you.