92847 2012-08-01 00:44:49 -0700 REGRESSION(r123893): Reftest mismatches are run through ImageDiff with 0.1 tolerance 2012-08-01 19:31:39 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified RESOLVED FIXED P2 Normal --- 1 zan dpranke abarth dpranke eric ojan rniwa tony webkit.review.bot oldest_to_newest 683941 0 zan 2012-08-01 00:44:49 -0700 After r123893[1] and the image diffing code refactoring, the default tolerance is set to 0.1 for the port when diffing images[2]. With the tolerance for diffing reftest output being 0, the expression on the next line[3] evaluates to 0.1. This means the ImageDiff operates with 0.1 tolerance, which can lead to false positives (i.e. ImageDiff reports that the two images' differences are within the allowed boundaries). I guess it should explicitly be checked whether the tolerance argument passed into base.diff_image is None and act accordingly. I don't know why the default value for the tolerance option is being overridden, though (if it is actually overridden rather than just ensured it exists). [1] - http://trac.webkit.org/changeset/123893 [2] - http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/layout_tests/port/base.py?rev=123893#L325 [2] - http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/layout_tests/port/base.py?rev=123893#L326 684835 1 155940 dpranke 2012-08-01 18:13:17 -0700 Created attachment 155940 Patch 684837 2 dpranke 2012-08-01 18:13:49 -0700 whoops. good catch. 684905 3 155940 webkit.review.bot 2012-08-01 19:31:35 -0700 Comment on attachment 155940 Patch Clearing flags on attachment: 155940 Committed r124401: <http://trac.webkit.org/changeset/124401> 684906 4 webkit.review.bot 2012-08-01 19:31:39 -0700 All reviewed patches have been landed. Closing bug. 155940 2012-08-01 18:13:17 -0700 2012-08-01 19:31:35 -0700 Patch bug-92847-20120801181259.patch text/plain 3197 dpranke U3VidmVyc2lvbiBSZXZpc2lvbjogMTI0Mzc4CmRpZmYgLS1naXQgYS9Ub29scy9DaGFuZ2VMb2cg Yi9Ub29scy9DaGFuZ2VMb2cKaW5kZXggZTlkNjZkYTMxNTc1Yjk2MzA4YTAwMTBjMWNkNDZiMmM5 NTNkMzAyYy4uNTBlZmFmYWY2ZmVmZDZhYzQ4ZjFmNjdmMTU2YTQ0ZmQ4YTYzZDRlYiAxMDA2NDQK LS0tIGEvVG9vbHMvQ2hhbmdlTG9nCisrKyBiL1Rvb2xzL0NoYW5nZUxvZwpAQCAtMSw1ICsxLDIw IEBACiAyMDEyLTA4LTAxICBEaXJrIFByYW5rZSAgPGRwcmFua2VAY2hyb21pdW0ub3JnPgogCisg ICAgICAgIFJFR1JFU1NJT04ocjEyMzg5Myk6IFJlZnRlc3QgbWlzbWF0Y2hlcyBhcmUgcnVuIHRo cm91Z2ggSW1hZ2VEaWZmIHdpdGggMC4xIHRvbGVyYW5jZQorICAgICAgICBodHRwczovL2J1Z3Mu d2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9OTI4NDcKKworICAgICAgICBSZXZpZXdlZCBieSBO T0JPRFkgKE9PUFMhKS4KKworICAgICAgICBJIHdhcyBmYWlsaW5nIHRvIGRpc3Rpbmd1aXNoIGJl dHdlZW4gJ05vbmUnIGFuZCAwIDooLiBGaXhlZCBhbmQgYWRkZWQgdGVzdHMuCisKKyAgICAgICAg KiBTY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9wb3J0L2Jhc2UucHk6CisgICAgICAgIChQ b3J0LmRpZmZfaW1hZ2UpOgorICAgICAgICAqIFNjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3Rz L3BvcnQvcG9ydF90ZXN0Y2FzZS5weToKKyAgICAgICAgKFBvcnRUZXN0Q2FzZS50ZXN0X2RpZmZf aW1hZ2UubWFrZV9wcm9jKToKKyAgICAgICAgKFBvcnRUZXN0Q2FzZS50ZXN0X2RpZmZfaW1hZ2Up OgorCisyMDEyLTA4LTAxICBEaXJrIFByYW5rZSAgPGRwcmFua2VAY2hyb21pdW0ub3JnPgorCiAg ICAgICAgIHRlc3Qtd2Via2l0cHk6IGNsZWFuIHVwIGhhbmRsaW5nIG9mIHRlc3RzIHRvIHNraXAK ICAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTkyOTA5CiAK ZGlmZiAtLWdpdCBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3BvcnQvYmFz ZS5weSBiL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3BvcnQvYmFzZS5weQpp bmRleCA1OTA5ZDdjZTYwMWMwZmJlMzAwMTU1ZmEyM2RjOThkY2E1ZmFmOWM3Li44OWEzNGU5Mjg4 NmI5OGNkY2M3MGU4MWJhNTNkOGNlMzExYmQzMzZlIDEwMDc1NQotLS0gYS9Ub29scy9TY3JpcHRz L3dlYmtpdHB5L2xheW91dF90ZXN0cy9wb3J0L2Jhc2UucHkKKysrIGIvVG9vbHMvU2NyaXB0cy93 ZWJraXRweS9sYXlvdXRfdGVzdHMvcG9ydC9iYXNlLnB5CkBAIC0zMjUsNyArMzI1LDggQEAgY2xh c3MgUG9ydChvYmplY3QpOgogICAgICAgICBpZiBub3Qgc2VsZi5faW1hZ2VfZGlmZmVyOgogICAg ICAgICAgICAgc2VsZi5faW1hZ2VfZGlmZmVyID0gaW1hZ2VfZGlmZi5JbWFnZURpZmZlcihzZWxm KQogICAgICAgICBzZWxmLnNldF9vcHRpb25fZGVmYXVsdCgndG9sZXJhbmNlJywgMC4xKQotICAg ICAgICB0b2xlcmFuY2UgPSB0b2xlcmFuY2Ugb3Igc2VsZi5nZXRfb3B0aW9uKCd0b2xlcmFuY2Un KQorICAgICAgICBpZiB0b2xlcmFuY2UgaXMgTm9uZToKKyAgICAgICAgICAgIHRvbGVyYW5jZSA9 IHNlbGYuZ2V0X29wdGlvbigndG9sZXJhbmNlJykKICAgICAgICAgcmV0dXJuIHNlbGYuX2ltYWdl X2RpZmZlci5kaWZmX2ltYWdlKGV4cGVjdGVkX2NvbnRlbnRzLCBhY3R1YWxfY29udGVudHMsIHRv bGVyYW5jZSkKIAogICAgIGRlZiBkaWZmX3RleHQoc2VsZiwgZXhwZWN0ZWRfdGV4dCwgYWN0dWFs X3RleHQsIGV4cGVjdGVkX2ZpbGVuYW1lLCBhY3R1YWxfZmlsZW5hbWUpOgpkaWZmIC0tZ2l0IGEv VG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvcG9ydC9wb3J0X3Rlc3RjYXNlLnB5 IGIvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvcG9ydC9wb3J0X3Rlc3RjYXNl LnB5CmluZGV4IGEzNjZlODk1MTlkMTJiZWM3Mzg5MTQ4MTVkZDEzZDM4MjdiNGYzN2YuLjU3Yjk4 MjJmYTE5NWFjZWU0MjZmMDA2YzFiNDhiM2RmMTQ0MTVmNDUgMTAwNzU1Ci0tLSBhL1Rvb2xzL1Nj cmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3BvcnQvcG9ydF90ZXN0Y2FzZS5weQorKysgYi9U b29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9wb3J0L3BvcnRfdGVzdGNhc2UucHkK QEAgLTI1NCwxMyArMjU0LDIwIEBAIGNsYXNzIFBvcnRUZXN0Q2FzZSh1bml0dGVzdC5UZXN0Q2Fz ZSk6CiAgICAgICAgIHNlbGYucHJvYyA9IE5vbmUKIAogICAgICAgICBkZWYgbWFrZV9wcm9jKHBv cnQsIG5tLCBjbWQsIGVudik6Ci0gICAgICAgICAgICBzZWxmLnByb2MgPSBNb2NrU2VydmVyUHJv Y2Vzcyhwb3J0LCBubSwgY21kLCBlbnYsIGxpbmVzPVsnZGlmZjogMTAwJSBmYWlsZWRcbiddKQor ICAgICAgICAgICAgc2VsZi5wcm9jID0gTW9ja1NlcnZlclByb2Nlc3MocG9ydCwgbm0sIGNtZCwg ZW52LCBsaW5lcz1bJ2RpZmY6IDEwMCUgZmFpbGVkXG4nLCAnZGlmZjogMTAwJSBmYWlsZWRcbidd KQogICAgICAgICAgICAgcmV0dXJuIHNlbGYucHJvYwogCiAgICAgICAgIHBvcnQuX3NlcnZlcl9w cm9jZXNzX2NvbnN0cnVjdG9yID0gbWFrZV9wcm9jCiAgICAgICAgIHBvcnQuc2V0dXBfdGVzdF9y dW4oKQogICAgICAgICBzZWxmLmFzc2VydEVxdWFscyhwb3J0LmRpZmZfaW1hZ2UoJ2ZvbycsICdi YXInKSwgKCcnLCAxMDAuMCkpCiAgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWxzKHNlbGYucHJvYy5j bWRbMTozXSwgWyItLXRvbGVyYW5jZSIsICIwLjEiXSkKKworICAgICAgICBzZWxmLmFzc2VydEVx dWFscyhwb3J0LmRpZmZfaW1hZ2UoJ2ZvbycsICdiYXInLCBOb25lKSwgKCcnLCAxMDAuMCkpCisg ICAgICAgIHNlbGYuYXNzZXJ0RXF1YWxzKHNlbGYucHJvYy5jbWRbMTozXSwgWyItLXRvbGVyYW5j ZSIsICIwLjEiXSkKKworICAgICAgICBzZWxmLmFzc2VydEVxdWFscyhwb3J0LmRpZmZfaW1hZ2Uo J2ZvbycsICdiYXInLCAwKSwgKCcnLCAxMDAuMCkpCisgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWxz KHNlbGYucHJvYy5jbWRbMTozXSwgWyItLXRvbGVyYW5jZSIsICIwIl0pCisKICAgICAgICAgcG9y dC5jbGVhbl91cF90ZXN0X3J1bigpCiAgICAgICAgIHNlbGYuYXNzZXJ0VHJ1ZShzZWxmLnByb2Mu c3RvcHBlZCkKICAgICAgICAgc2VsZi5hc3NlcnRFcXVhbHMocG9ydC5faW1hZ2VfZGlmZmVyLCBO b25lKQo=