Add pixel tolerance to some compositing tests

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Simon Fraser (smfr)]

Add pixel tolerance to some compositing tests

[Simon Fraser (smfr)]

Created attachment 449710 [details]
Patch

[Darin Adler]

Comment on attachment 449710 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=449710&action=review

rs=me with a small set of clarifying questions

> LayoutTests/ChangeLog:8
> +        Add pixel tolerance to tests whose output is slightly different when accelerated drawing

What’s the rationale here? How much pixel tolerance is too much? Are there some tests where adding pixel tolerance would be a bad idea? Is it a good practice to add pixel tolerance needed only on iOS simulator with accelerated drawing as a cross platform parameter? Is this a forever good decision, or would we want to return and lower the tolerance later?

[Simon Fraser (smfr)]

(In reply to Darin Adler from comment #2)
> Comment on attachment 449710 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=449710&action=review
> 
> rs=me with a small set of clarifying questions
> 
> > LayoutTests/ChangeLog:8
> > +        Add pixel tolerance to tests whose output is slightly different when accelerated drawing
> 
> What’s the rationale here? How much pixel tolerance is too much? Are there
> some tests where adding pixel tolerance would be a bad idea? Is it a good
> practice to add pixel tolerance needed only on iOS simulator with
> accelerated drawing as a cross platform parameter? Is this a forever good
> decision, or would we want to return and lower the tolerance later?

The numbers are based on the pixel tolerance results from a run with iOS accelerated drawing enabled (bug 231828). I only add tolerance to tests where the image failure is obviously the result of differences in rendering implementation (e.g. antialiasing fuzz) and not some real failure.

Currently we don't have a way to express per-platform tolerance. I expect some future version of this feature will move the tolerance data outside of the test file (see the webkit-dev discussion from 2021-11).

[EWS]

Committed r288449 (246338@main): <https://commits.webkit.org/246338@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 449710 [details].

[Radar WebKit Bug Importer]

<rdar://problem/87974349>