Bug 232523 – Expose fuzzy match data in layout test results

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Simon Fraser (smfr)]

Expose fuzzy match data in layout test results

[Simon Fraser (smfr)]

Created attachment 442898 [details]
Patch

[Simon Fraser (smfr)]

Created attachment 442900 [details]
Screenshot of the UI

[Simon Fraser (smfr)]

Created attachment 442901 [details]
Patch

[Simon Fraser (smfr)]

Created attachment 442902 [details]
Screenshot of the UI

[Jonathan Bedard]

Comment on attachment 442901 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=442901&action=review

The webkitpy failure (on Python 2) appears to be genuine (webkitpy.layout_tests.run_webkit_tests_integrationtest.RunTest.test_pixel_test_directories is the test)

> Tools/ChangeLog:9
> +        in the form "image_difference": { "max_difference": 13, "total_pixels": 167 }, and

Want to call out that this information is not included in results database uploads...I think that's fine, including it there to be displayed in a reasonable way would require some modification of the service.

[Simon Fraser (smfr)]

Created attachment 443170 [details]
For EWS

[Simon Fraser (smfr)]

Created attachment 443173 [details]
Patch

[Simon Fraser (smfr)]

(In reply to Jonathan Bedard from comment #5)
> Comment on attachment 442901 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=442901&action=review
> 
> The webkitpy failure (on Python 2) appears to be genuine
> (webkitpy.layout_tests.run_webkit_tests_integrationtest.RunTest.
> test_pixel_test_directories is the test)
> 
> > Tools/ChangeLog:9
> > +        in the form "image_difference": { "max_difference": 13, "total_pixels": 167 }, and
> 
> Want to call out that this information is not included in results database
> uploads...I think that's fine, including it there to be displayed in a
> reasonable way would require some modification of the service.

These failures turned out to be about JSON property serialization order (the test was relying on ordering).

[Simon Fraser (smfr)]

(In reply to Jonathan Bedard from comment #5)
> Comment on attachment 442901 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=442901&action=review
> 
> The webkitpy failure (on Python 2) appears to be genuine
> (webkitpy.layout_tests.run_webkit_tests_integrationtest.RunTest.
> test_pixel_test_directories is the test)
> 
> > Tools/ChangeLog:9
> > +        in the form "image_difference": { "max_difference": 13, "total_pixels": 167 }, and
> 
> Want to call out that this information is not included in results database
> uploads.

This data is in the json, so would that not be included in database uploads? I don't know how that works.

[Jonathan Bedard]

(In reply to Simon Fraser (smfr) from comment #9)
> (In reply to Jonathan Bedard from comment #5)
> > Comment on attachment 442901 [details]
> > Patch
> > 
> > View in context:
> > https://bugs.webkit.org/attachment.cgi?id=442901&action=review
> > 
> > The webkitpy failure (on Python 2) appears to be genuine
> > (webkitpy.layout_tests.run_webkit_tests_integrationtest.RunTest.
> > test_pixel_test_directories is the test)
> > 
> > > Tools/ChangeLog:9
> > > +        in the form "image_difference": { "max_difference": 13, "total_pixels": 167 }, and
> > 
> > Want to call out that this information is not included in results database
> > uploads.
> 
> This data is in the json, so would that not be included in database uploads?
> I don't know how that works.

Right! Just wanted to call that out so anyone looking at this in the future doesn't have to guess.

[EWS]

Committed r285213 (243838@main): <https://commits.webkit.org/243838@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 443173 [details].

[Radar WebKit Bug Importer]

<rdar://problem/84979414>