94746 2012-08-22 13:47:39 -0700 decide what (and how) we should set the tolerance for ref test pixel compares and test for that 2012-08-30 13:22:17 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified RESOLVED FIXED NRWT P2 Normal --- 1 dpranke dpranke abarth eric ojan rniwa simon.fraser tony webkit.review.bot zan oldest_to_newest 702295 0 dpranke 2012-08-22 13:47:39 -0700 See comments in https://bugs.webkit.org/show_bug.cgi?id=94704 for background ... it's perhaps not clear if we should use the default tolerance the port specifies for pixel tests also for ref tests, or if ref tests should always require an exact match (tolerance == 0 or just hash compares), or if we should have a separate default-tolerance-for-ref-tests parameter. Once we agree on this, we should make sure this is implemented and write a test to check the behavior (we don't have a test today). 702299 1 ojan 2012-08-22 13:52:48 -0700 I don't see why we'd ever want a tolerance other than 0 for reftests. The point of tolerance is to handle things like different text anti-aliasing across OS releases. None of these sort of things apply to reftests. 702314 2 rniwa 2012-08-22 14:00:58 -0700 Yeah, it seems like we shouldn't need any tolerance for ref tests. 702335 3 dpranke 2012-08-22 14:15:46 -0700 I don't have any real argument against using 0. Simon, anyone else from Apple, any objections? 702386 4 simon.fraser 2012-08-22 14:56:48 -0700 No objection, but I wouldn't be surprised if some ref tests are "close but not exact" and fail with a zero tolerance. We'll have to try it and see. 702389 5 rniwa 2012-08-22 14:59:27 -0700 I guess 1px rounding differences are common (e.g. inline-block vs. inline) in text rendering. 702391 6 dpranke 2012-08-22 15:00:48 -0700 it would seem a bit odd to me to have reftests that attempt to render the same string of text two different ways and hope that they would be the same, but maybe I'm not being creative enough. Do we have many of those? 702394 7 rniwa 2012-08-22 15:03:32 -0700 (In reply to comment #6) > it would seem a bit odd to me to have reftests that attempt to render the same string of text two different ways and hope that they would be the same, but maybe I'm not being creative enough. Do we have many of those? I've hit that when I was converting a test for bdi because inline-block and inline have a different semantics in UBA. 708904 8 161534 dpranke 2012-08-30 12:35:04 -0700 Created attachment 161534 Patch 708905 9 dpranke 2012-08-30 12:35:32 -0700 Looks like there's general consensus that we should require exact matches. I'm adding an assertion to the test cases to enforce this. 708957 10 161534 webkit.review.bot 2012-08-30 13:22:13 -0700 Comment on attachment 161534 Patch Clearing flags on attachment: 161534 Committed r127180: <http://trac.webkit.org/changeset/127180> 708958 11 webkit.review.bot 2012-08-30 13:22:17 -0700 All reviewed patches have been landed. Closing bug. 161534 2012-08-30 12:35:04 -0700 2012-08-30 13:22:13 -0700 Patch bug-94746-20120830123453.patch text/plain 1570 dpranke U3VidmVyc2lvbiBSZXZpc2lvbjogMTI3MDU4CmRpZmYgLS1naXQgYS9Ub29scy9DaGFuZ2VMb2cg Yi9Ub29scy9DaGFuZ2VMb2cKaW5kZXggNDljMGE3ZDNhZmZjNmE3NmYwMzkzNmE2Y2RjYzgzNjVj ODYzZWE2Ny4uNTRjMWY0ZWNiMTk5YzlhMzMzZDg2MGU2YzdkZjZkYzA2NzU4OGMzZCAxMDA2NDQK LS0tIGEvVG9vbHMvQ2hhbmdlTG9nCisrKyBiL1Rvb2xzL0NoYW5nZUxvZwpAQCAtMSwzICsxLDE2 IEBACisyMDEyLTA4LTMwICBEaXJrIFByYW5rZSAgPGRwcmFua2VAY2hyb21pdW0ub3JnPgorCisg ICAgICAgIGRlY2lkZSB3aGF0IChhbmQgaG93KSB3ZSBzaG91bGQgc2V0IHRoZSB0b2xlcmFuY2Ug Zm9yIHJlZiB0ZXN0IHBpeGVsIGNvbXBhcmVzIGFuZCB0ZXN0IGZvciB0aGF0CisgICAgICAgIGh0 dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD05NDc0NgorCisgICAgICAgIFJl dmlld2VkIGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgIEFkZCBhbiBhc3NlcnRpb24gdG8g ZGlmZl9pbWFnZSgpIHRvIGNoZWNrIHRoYXQgd2UgYXJlIHBhc3NpbmcKKyAgICAgICAgdG9sZXJh bmNlPTAgZXhwbGljaXRseSB3aGVuIGRpZmZpbmcgcmVmIHRlc3QgcmVzdWx0cy4KKworICAgICAg ICAqIFNjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3BvcnQvdGVzdC5weToKKyAgICAgICAg KFRlc3RQb3J0LmRpZmZfaW1hZ2UpOgorCiAyMDEyLTA4LTI5ICBEaXJrIFByYW5rZSAgPGRwcmFu a2VAY2hyb21pdW0ub3JnPgogCiAgICAgICAgIHdlYmtpdC1wYXRjaCByZWJhc2VsaW5lLWV4cGVj dGF0aW9ucyB3cm9uZ2x5IHRvdWNoZXMgb3RoZXIgcGxhdGZvcm1zJyBUZXN0RXhwZWN0YXRpb25z CmRpZmYgLS1naXQgYS9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9wb3J0L3Rl c3QucHkgYi9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9wb3J0L3Rlc3QucHkK aW5kZXggY2NjYjlhYzFkOTA2ZjZlOGU4MGU1OTRjYjNhMjQ3MDIzMzRlYzJhYS4uZTQ2MTVkNTcw YmJiOTViOWFiMDFkM2I0MzRmMzhlMzQxYTkyZjAyOCAxMDA2NDQKLS0tIGEvVG9vbHMvU2NyaXB0 cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvcG9ydC90ZXN0LnB5CisrKyBiL1Rvb2xzL1NjcmlwdHMv d2Via2l0cHkvbGF5b3V0X3Rlc3RzL3BvcnQvdGVzdC5weQpAQCAtNDA1LDYgKzQwNSw4IEBAIGNs YXNzIFRlc3RQb3J0KFBvcnQpOgogCiAgICAgZGVmIGRpZmZfaW1hZ2Uoc2VsZiwgZXhwZWN0ZWRf Y29udGVudHMsIGFjdHVhbF9jb250ZW50cywgdG9sZXJhbmNlPU5vbmUpOgogICAgICAgICBkaWZm ZWQgPSBhY3R1YWxfY29udGVudHMgIT0gZXhwZWN0ZWRfY29udGVudHMKKyAgICAgICAgaWYgJ3Jl ZicgaW4gZXhwZWN0ZWRfY29udGVudHM6CisgICAgICAgICAgICBhc3NlcnQgdG9sZXJhbmNlID09 IDAKICAgICAgICAgaWYgZGlmZmVkOgogICAgICAgICAgICAgcmV0dXJuICgiPCAlc1xuLS0tXG4+ ICVzXG4iICUgKGV4cGVjdGVkX2NvbnRlbnRzLCBhY3R1YWxfY29udGVudHMpLCAxLCBOb25lKQog ICAgICAgICByZXR1cm4gKE5vbmUsIDAsIE5vbmUpCg==