127332 2014-01-20 20:15:40 -0800 Shouldn't Failure expectation work for image-only failures? 2016-04-11 10:18:23 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified REOPENED P2 Normal --- 1 ap webkit-unassigned dbates dburkart dean_johnson dpranke matthew_hanson rniwa zalan oldest_to_newest 970151 0 ap 2014-01-20 20:15:40 -0800 Failure allows for most kinds of failures, namely TEXT, IMAGE_PLUS_TEXT, and AUDIO. But why not image-only failures? Notably, one would expect that it works for reftests, I think. 970161 1 rniwa 2014-01-20 20:52:24 -0800 Yeah, I always disliked this. Failure should include ImageOnlyFailure. 970503 2 dpranke 2014-01-21 17:57:00 -0800 There were discussions on this when we were talking about changing the syntax back in 6/12 (and then in 9/12 when the changes were landing). The general theory (I think) was that if a test is failing the pixel results and starts failing the text results as well, that almost certainly is a regression that should be noticed (and, conversely, if a test is now only failing the pixel tests, that's an improvement that should be noticed). There was also the belief that this mattered more to the Chromium devs than others :). Further, there were also debates as to whether reftests should be ImageOnlyFailure or just Failure. I don't think this one matters much one way or another. 970504 3 dpranke 2014-01-21 17:57:15 -0800 You are, of course, free to change it to however you'd like it to work these days. 1182630 4 rniwa 2016-04-11 01:16:10 -0700 Won't fixing this. Please re-open this bug if you feel strongly about it. 1182716 5 ap 2016-04-11 09:52:11 -0700 Yes, people still get confused about this. In fact, the latest I heard someone complain was last week. 1182719 6 rniwa 2016-04-11 09:58:44 -0700 (In reply to comment #5) > Yes, people still get confused about this. In fact, the latest I heard > someone complain was last week. Okay. This should be easy enough for someone in the tools' team to fix. I can give some guidance as needed.