138070 2014-10-25 13:54:11 -0700 Test regressions are not detected when image result is missing 2014-10-25 16:29:53 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified RESOLVED FIXED P2 Normal --- 1 ap ap commit-queue dbates glenn jake.nielsen.webkit rniwa simon.fraser thorton oldest_to_newest 1044139 0 ap 2014-10-25 13:54:11 -0700 When a test fails on first try, some bots retry it with pixel testing enabled. For tests that don't have pixel results checked in, this results in a "TEXT MISSING" failure (TEXT on first try, MISSING on second try). These are not reported as true regressions by run-webkit-tests, buildbot, or dashboard. See e.g. https://build.webkit.org/builders/Apple%20Yosemite%20Release%20WK2%20(Tests)/builds/173 - there are 6 editing/selection tests that fail, but the test run is not red. 1044140 1 240457 ap 2014-10-25 13:58:37 -0700 Created attachment 240457 proposed fix 1044141 2 240457 simon.fraser 2014-10-25 14:11:51 -0700 Comment on attachment 240457 proposed fix Nice. 1044143 3 240457 commit-queue 2014-10-25 16:29:50 -0700 Comment on attachment 240457 proposed fix Clearing flags on attachment: 240457 Committed r175204: <http://trac.webkit.org/changeset/175204> 1044144 4 commit-queue 2014-10-25 16:29:53 -0700 All reviewed patches have been landed. Closing bug. 240457 2014-10-25 13:58:37 -0700 2014-10-25 16:29:50 -0700 proposed fix TextMissing.txt text/plain 5861 ap SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDE3NTIwMikKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE4IEBACisyMDE0LTEwLTI1ICBBbGV4ZXkgUHJvc2t1cnlha292ICA8YXBAYXBwbGUuY29t PgorCisgICAgICAgIFRlc3QgcmVncmVzc2lvbnMgYXJlIG5vdCBkZXRlY3RlZCB3aGVuIGltYWdl IHJlc3VsdCBpcyBtaXNzaW5nCisgICAgICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3df YnVnLmNnaT9pZD0xMzgwNzAKKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4K KworICAgICAgICAqIFNjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL21vZGVscy90ZXN0X3J1 bl9yZXN1bHRzLnB5OgorICAgICAgICAqIFNjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3Zp ZXdzL2J1aWxkYm90X3Jlc3VsdHMucHk6CisgICAgICAgIENvdW50IHRoZXNlIGFzIHJlZ3Jlc3Np b25zLCBub3QgYXMgZmxha3kgdGVzdHMuCisKKyAgICAgICAgKiBTY3JpcHRzL3dlYmtpdHB5L3Bv cnQvdGVzdC5weToKKyAgICAgICAgKiBTY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9ydW5f d2Via2l0X3Rlc3RzX2ludGVncmF0aW9udGVzdC5weToKKyAgICAgICAgVGVzdCBpdC4KKwogMjAx NC0xMC0yNCAgVGltb3RoeSBIb3J0b24gIDx0aW1vdGh5X2hvcnRvbkBhcHBsZS5jb20+CiAKICAg ICAgICAgQWRkIENvbnJhZCBTaHVsdHogdG8gdGhlIGNvbnRyaWJ1dG9ycyBsaXN0LgpJbmRleDog VG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvcnVuX3dlYmtpdF90ZXN0c19pbnRl Z3JhdGlvbnRlc3QucHkKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlv dXRfdGVzdHMvcnVuX3dlYmtpdF90ZXN0c19pbnRlZ3JhdGlvbnRlc3QucHkJKHJldmlzaW9uIDE3 NTE5OCkKKysrIFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3J1bl93ZWJraXRf dGVzdHNfaW50ZWdyYXRpb250ZXN0LnB5CSh3b3JraW5nIGNvcHkpCkBAIC02NzEsNiArNjcxLDIy IEBAIGNsYXNzIFJ1blRlc3QodW5pdHRlc3QuVGVzdENhc2UsIFN0cmVhbVQKICAgICAgICAgc2Vs Zi5hc3NlcnRGYWxzZShqc29uWyJwaXhlbF90ZXN0c19lbmFibGVkIl0pCiAgICAgICAgIHNlbGYu YXNzZXJ0RXF1YWwoZGV0YWlscy5lbmFibGVkX3BpeGVsX3Rlc3RzX2luX3JldHJ5LCBUcnVlKQog CisgICAgZGVmIHRlc3RfZmFpbGVkX3RleHRfd2l0aF9taXNzaW5nX3BpeGVsX3Jlc3VsdHNfb25f cmV0cnkoc2VsZik6CisgICAgICAgICMgVGVzdCB3aGF0IGhhcHBlbnMgd2hlbiBwaXhlbCByZXN1 bHRzIGFyZSBtaXNzaW5nIG9uIHJldHJ5LgorICAgICAgICBob3N0ID0gTW9ja0hvc3QoKQorICAg ICAgICBkZXRhaWxzLCBlcnIsIF8gPSBsb2dnaW5nX3J1bihbJy0tbm8tc2hvdy1yZXN1bHRzJywK KyAgICAgICAgICAgICctLW5vLW5ldy10ZXN0LXJlc3VsdHMnLCAnLS1uby1waXhlbC10ZXN0cycs CisgICAgICAgICAgICAnZmFpbHVyZXMvdW5leHBlY3RlZC90ZXh0LWltYWdlLW1pc3NpbmcuaHRt bCddLAorICAgICAgICAgICAgdGVzdHNfaW5jbHVkZWQ9VHJ1ZSwgaG9zdD1ob3N0KQorICAgICAg ICBmaWxlX2xpc3QgPSBob3N0LmZpbGVzeXN0ZW0ud3JpdHRlbl9maWxlcy5rZXlzKCkKKyAgICAg ICAgc2VsZi5hc3NlcnRFcXVhbChkZXRhaWxzLmV4aXRfY29kZSwgMSkKKyAgICAgICAgZXhwZWN0 ZWRfdG9rZW4gPSAnInVuZXhwZWN0ZWQiOnsidGV4dC1pbWFnZS1taXNzaW5nLmh0bWwiOnsicmVw b3J0IjoiUkVHUkVTU0lPTiIsImV4cGVjdGVkIjoiUEFTUyIsImFjdHVhbCI6IlRFWFQgTUlTU0lO RyIsImlzX21pc3NpbmdfaW1hZ2UiOnRydWV9fScKKyAgICAgICAganNvbl9zdHJpbmcgPSBob3N0 LmZpbGVzeXN0ZW0ucmVhZF90ZXh0X2ZpbGUoJy90bXAvbGF5b3V0LXRlc3QtcmVzdWx0cy9mdWxs X3Jlc3VsdHMuanNvbicpCisgICAgICAgIHNlbGYuYXNzZXJ0VHJ1ZShqc29uX3N0cmluZy5maW5k KGV4cGVjdGVkX3Rva2VuKSAhPSAtMSkKKyAgICAgICAgc2VsZi5hc3NlcnRUcnVlKGpzb25fc3Ry aW5nLmZpbmQoJyJudW1fcmVncmVzc2lvbnMiOjEnKSAhPSAtMSkKKyAgICAgICAgc2VsZi5hc3Nl cnRUcnVlKGpzb25fc3RyaW5nLmZpbmQoJyJudW1fZmxha3kiOjAnKSAhPSAtMSkKKyAgICAgICAg c2VsZi5hc3NlcnRUcnVlKGpzb25fc3RyaW5nLmZpbmQoJyJudW1fbWlzc2luZyI6MScpICE9IC0x KQorCiAgICAgZGVmIHRlc3RfcmV0cnlpbmdfdXNlc19yZXRyaWVzX2RpcmVjdG9yeShzZWxmKToK ICAgICAgICAgaG9zdCA9IE1vY2tIb3N0KCkKICAgICAgICAgZGV0YWlscywgZXJyLCBfID0gbG9n Z2luZ19ydW4oWyctLWRlYnVnLXJ3dC1sb2dnaW5nJywgJ2ZhaWx1cmVzL3VuZXhwZWN0ZWQvdGV4 dC1pbWFnZS1jaGVja3N1bS5odG1sJ10sIHRlc3RzX2luY2x1ZGVkPVRydWUsIGhvc3Q9aG9zdCkK SW5kZXg6IFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL21vZGVscy90ZXN0X3J1 bl9yZXN1bHRzLnB5Cj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0 X3Rlc3RzL21vZGVscy90ZXN0X3J1bl9yZXN1bHRzLnB5CShyZXZpc2lvbiAxNzUxOTgpCisrKyBU b29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9tb2RlbHMvdGVzdF9ydW5fcmVzdWx0 cy5weQkod29ya2luZyBjb3B5KQpAQCAtMTkyLDcgKzE5Miw5IEBAIGRlZiBzdW1tYXJpemVfcmVz dWx0cyhwb3J0X29iaiwgZXhwZWN0YXQKICAgICAgICAgICAgIGVsaWYgcmV0cnlfcmVzdWx0czoK ICAgICAgICAgICAgICAgICByZXRyeV9yZXN1bHRfdHlwZSA9IHJldHJ5X3Jlc3VsdHMudW5leHBl Y3RlZF9yZXN1bHRzX2J5X25hbWVbdGVzdF9uYW1lXS50eXBlCiAgICAgICAgICAgICAgICAgaWYg cmVzdWx0X3R5cGUgIT0gcmV0cnlfcmVzdWx0X3R5cGU6Ci0gICAgICAgICAgICAgICAgICAgIGlm IGVuYWJsZWRfcGl4ZWxfdGVzdHNfaW5fcmV0cnkgYW5kIHJlc3VsdF90eXBlID09IHRlc3RfZXhw ZWN0YXRpb25zLlRFWFQgYW5kIHJldHJ5X3Jlc3VsdF90eXBlID09IHRlc3RfZXhwZWN0YXRpb25z LklNQUdFX1BMVVNfVEVYVDoKKyAgICAgICAgICAgICAgICAgICAgaWYgZW5hYmxlZF9waXhlbF90 ZXN0c19pbl9yZXRyeSBhbmQgcmVzdWx0X3R5cGUgPT0gdGVzdF9leHBlY3RhdGlvbnMuVEVYVCBh bmQgKHJldHJ5X3Jlc3VsdF90eXBlID09IHRlc3RfZXhwZWN0YXRpb25zLklNQUdFX1BMVVNfVEVY VCBvciByZXRyeV9yZXN1bHRfdHlwZSA9PSB0ZXN0X2V4cGVjdGF0aW9ucy5NSVNTSU5HKToKKyAg ICAgICAgICAgICAgICAgICAgICAgIGlmIHJldHJ5X3Jlc3VsdF90eXBlID09IHRlc3RfZXhwZWN0 YXRpb25zLk1JU1NJTkc6CisgICAgICAgICAgICAgICAgICAgICAgICAgICAgbnVtX21pc3Npbmcg Kz0gMQogICAgICAgICAgICAgICAgICAgICAgICAgbnVtX3JlZ3Jlc3Npb25zICs9IDEKICAgICAg ICAgICAgICAgICAgICAgICAgIHRlc3RfZGljdFsncmVwb3J0J10gPSAnUkVHUkVTU0lPTicKICAg ICAgICAgICAgICAgICAgICAgZWxzZToKSW5kZXg6IFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5 b3V0X3Rlc3RzL3ZpZXdzL2J1aWxkYm90X3Jlc3VsdHMucHkKPT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMv U2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvdmlld3MvYnVpbGRib3RfcmVzdWx0cy5weQko cmV2aXNpb24gMTc1MTk4KQorKysgVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMv dmlld3MvYnVpbGRib3RfcmVzdWx0cy5weQkod29ya2luZyBjb3B5KQpAQCAtMTE1LDcgKzExNSw3 IEBAIGNsYXNzIEJ1aWxkQm90UHJpbnRlcihvYmplY3QpOgogICAgICAgICAgICAgICAgICAgICBh ZGRfdG9fZGljdF9vZl9saXN0cyhwYXNzZXMsICdFeHBlY3RlZCB0byB0aW1lb3V0LCBidXQgcGFz c2VkJywgdGVzdCkKICAgICAgICAgICAgICAgICBlbHNlOgogICAgICAgICAgICAgICAgICAgICBh ZGRfdG9fZGljdF9vZl9saXN0cyhwYXNzZXMsICdFeHBlY3RlZCB0byBmYWlsLCBidXQgcGFzc2Vk JywgdGVzdCkKLSAgICAgICAgICAgIGVsaWYgZW5hYmxlZF9waXhlbF90ZXN0c19pbl9yZXRyeSBh bmQgYWN0dWFsID09IFsnVEVYVCcsICdJTUFHRStURVhUJ106CisgICAgICAgICAgICBlbGlmIGVu YWJsZWRfcGl4ZWxfdGVzdHNfaW5fcmV0cnkgYW5kIChhY3R1YWwgPT0gWydURVhUJywgJ0lNQUdF K1RFWFQnXSBvciBhY3R1YWwgPT0gWydURVhUJywgJ01JU1NJTkcnXSk6CiAgICAgICAgICAgICAg ICAgYWRkX3RvX2RpY3Rfb2ZfbGlzdHMocmVncmVzc2lvbnMsIGFjdHVhbFswXSwgdGVzdCkKICAg ICAgICAgICAgIGVsaWYgbGVuKGFjdHVhbCkgPiAxOgogICAgICAgICAgICAgICAgICMgV2UgZ3Jv dXAgZmxha3kgdGVzdHMgYnkgdGhlIGZpcnN0IGFjdHVhbCByZXN1bHQgd2UgZ290LgpJbmRleDog VG9vbHMvU2NyaXB0cy93ZWJraXRweS9wb3J0L3Rlc3QucHkKPT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMv U2NyaXB0cy93ZWJraXRweS9wb3J0L3Rlc3QucHkJKHJldmlzaW9uIDE3NTE5OCkKKysrIFRvb2xz L1NjcmlwdHMvd2Via2l0cHkvcG9ydC90ZXN0LnB5CSh3b3JraW5nIGNvcHkpCkBAIC0xMDAsMTEg KzEwMCwxMSBAQCBjbGFzcyBUZXN0TGlzdChvYmplY3QpOgogIwogIyBUaGVzZSBudW1iZXJzIG1h eSBuZWVkIHRvIGJlIHVwZGF0ZWQgd2hlbmV2ZXIgd2UgYWRkIG9yIGRlbGV0ZSB0ZXN0cy4KICMK LVRPVEFMX1RFU1RTID0gMTA2CitUT1RBTF9URVNUUyA9IDEwNwogVE9UQUxfU0tJUFMgPSAyOAog VE9UQUxfUkVUUklFUyA9IDE0CiAKLVVORVhQRUNURURfUEFTU0VTID0gNgorVU5FWFBFQ1RFRF9Q QVNTRVMgPSA3CiBVTkVYUEVDVEVEX0ZBSUxVUkVTID0gMTcKIAogZGVmIHVuaXRfdGVzdF9saXN0 KCk6CkBAIC0xNjcsNiArMTY3LDkgQEAgbGF5ZXIgYXQgKDAsMCkgc2l6ZSA4MDB4MzQKICAgICAg ICAgICAgICAgYWN0dWFsX3RleHQ9J3RleHQtaW1hZ2UtY2hlY2tzdW1fZmFpbC10eHQnLAogICAg ICAgICAgICAgICBhY3R1YWxfaW1hZ2U9J3RleHQtaW1hZ2UtY2hlY2tzdW1fZmFpbC1wbmd0RVh0 Y2hlY2tzdW1ceDAwY2hlY2tzdW1fZmFpbCcsCiAgICAgICAgICAgICAgIGFjdHVhbF9jaGVja3N1 bT0ndGV4dC1pbWFnZS1jaGVja3N1bV9mYWlsLWNoZWNrc3VtJykKKyAgICB0ZXN0cy5hZGQoJ2Zh aWx1cmVzL3VuZXhwZWN0ZWQvdGV4dC1pbWFnZS1taXNzaW5nLmh0bWwnLAorICAgICAgICAgICAg ICBhY3R1YWxfdGV4dD0ndGV4dC1pbWFnZS1jaGVja3N1bV9mYWlsLXR4dCcsCisgICAgICAgICAg ICAgIGV4cGVjdGVkX2ltYWdlPU5vbmUpCiAgICAgdGVzdHMuYWRkKCdmYWlsdXJlcy91bmV4cGVj dGVkL2NoZWNrc3VtLXdpdGgtbWF0Y2hpbmctaW1hZ2UuaHRtbCcsCiAgICAgICAgICAgICAgIGFj dHVhbF9jaGVja3N1bT0ndGV4dC1pbWFnZS1jaGVja3N1bV9mYWlsLWNoZWNrc3VtJykKICAgICB0 ZXN0cy5hZGQoJ2ZhaWx1cmVzL3VuZXhwZWN0ZWQvc2tpcF9wYXNzLmh0bWwnKQo=