36619 2010-03-25 14:31:43 -0700 new-run-webkit-tests should use a test_expectations file in addition to Skipped files so we can handle pixel failures 2010-03-26 17:15:30 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) PC OS X 10.5 RESOLVED FIXED P2 Normal --- 1 dpranke dpranke cjerdonek eric ojan tony oldest_to_newest 204222 0 dpranke 2010-03-25 14:31:43 -0700 Skipped files give us no way to track pixel failures separately from other types of failures. If we start using a test_expectations.txt file we can track these and get a clean test run both with and without pixel tests enabled. 204227 1 51677 dpranke 2010-03-25 14:41:05 -0700 Created attachment 51677 patch 204228 2 51677 eric 2010-03-25 14:42:29 -0700 Comment on attachment 51677 patch Maybe that should go in mac-leopard? LGTM. 204238 3 dpranke 2010-03-25 14:58:01 -0700 test_expectations files should generally be as portable as possible, because maintaining multiple versions is a hassle, and one hopes that most bugs are cross-platform/cross-version, rather than version-specific. 204870 4 dpranke 2010-03-26 17:15:30 -0700 Committed r56648: <http://trac.webkit.org/changeset/56648> 51677 2010-03-25 14:41:05 -0700 2010-03-25 14:42:29 -0700 patch wk36619_test_expectations.txt text/plain 7052 dpranke ZGlmZiAtLWdpdCBhL0xheW91dFRlc3RzL0NoYW5nZUxvZyBiL0xheW91dFRlc3RzL0NoYW5nZUxv ZwppbmRleCAzYjllZTQ4Li5kZjg3ZTEyIDEwMDY0NAotLS0gYS9MYXlvdXRUZXN0cy9DaGFuZ2VM b2cKKysrIGIvTGF5b3V0VGVzdHMvQ2hhbmdlTG9nCkBAIC0xLDMgKzEsMTQgQEAKKzIwMTAtMDMt MjUgIERpcmsgUHJhbmtlICA8ZHByYW5rZUBjaHJvbWl1bS5vcmc+CisKKyAgICAgICAgUmV2aWV3 ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKKyAgICAgICAgQWRkIGEgdGVzdF9leHBlY3RhdGlvbnMu dHh0IGZpbGUgZm9yIHRoZSBtYWMgcGxhdGZvcm0gc28KKyAgICAgICAgbmV3LXJ1bi13ZWJraXQt dGVzdHMgY2FuIHRyYWNrIElNQUdFIGZhaWx1cmVzLgorCisgICAgICAgIGh0dHBzOi8vYnVncy53 ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD0zNjYxOQorCisgICAgICAgICogcGxhdGZvcm0vbWFj L3Rlc3RfZXhwZWN0YXRpb25zLnR4dDogQWRkZWQuCisKIDIwMTAtMDMtMjIgIERhcmluIEZpc2hl ciAgPGRhcmluQGNocm9taXVtLm9yZz4KIAogICAgICAgICBSZXZpZXdlZCBieSBCcmFkeSBFaWRz b24uCmRpZmYgLS1naXQgYS9MYXlvdXRUZXN0cy9wbGF0Zm9ybS9tYWMvdGVzdF9leHBlY3RhdGlv bnMudHh0IGIvTGF5b3V0VGVzdHMvcGxhdGZvcm0vbWFjL3Rlc3RfZXhwZWN0YXRpb25zLnR4dApu ZXcgZmlsZSBtb2RlIDEwMDY0NAppbmRleCAwMDAwMDAwLi44NjY0MDg1Ci0tLSAvZGV2L251bGwK KysrIGIvTGF5b3V0VGVzdHMvcGxhdGZvcm0vbWFjL3Rlc3RfZXhwZWN0YXRpb25zLnR4dApAQCAt MCwwICsxLDU3IEBACisvLyBUaGlzIGZpbGUgY29uc2lzdHMgb2YgbGluZXMgd2l0aCBzcGVjaWZp Y2F0aW9ucyBvZiB3aGF0CisvLyB0byBleHBlY3QgZnJvbSBsYXlvdXQgdGVzdCBjYXNlcy4gVGhl IHRlc3QgY2FzZXMgY2FuIGJlIGRpcmVjdG9yaWVzCisvLyBpbiB3aGljaCBjYXNlIHRoZSBleHBl Y3RhdGlvbnMgYXBwbHkgdG8gYWxsIHRlc3QgY2FzZXMgaW4gdGhhdAorLy8gZGlyZWN0b3J5IGFu ZCBhbnkgc3ViZGlyZWN0b3J5LiBUaGUgZm9ybWF0IG9mIHRoZSBmaWxlIGlzIGFsb25nIHRoZQor Ly8gbGluZXMgb2Y6CisvLworLy8gZmFzdC9qcy9maXhtZS5qcyA9IEZBSUwKKy8vIGZhc3QvanMv Zmxha3kuanMgPSBGQUlMIFBBU1MKKy8vIGZhc3QvanMvY3Jhc2guanMgPSBDUkFTSCBUSU1FT1VU IEZBSUwgUEFTUworLy8KKy8vIFRoZSBmb3JtYXQgb2YgYSB0ZXN0IGxpbmUgaXMgYXMgZm9sbG93 czoKKy8vIE1FVEFEQVRBIDogcmVsYXRpdmUvcGF0aC90by90ZXN0Lmh0bWwgPSBFWFBFQ1RBVElP TlMKKy8vCisvLyBWYWxpZCB2YWx1ZXMgZm9yIE1FVEFEQVRBOgorLy8KKy8vIEJVR1swLTldKzog U2VlIHRoaXMgYnVnIGZvciBtb3JlIGluZm9ybWF0aW9uLiBFdmVyeSB0ZXN0IHRoYXQgaXNuJ3Qg bWFya2VkCisvLyAgICAgICAgICAgIFdPTlRGSVggc2hvdWxkIGhhdmUgYSBCVUcgYW5ub3RhdGlv bi4KKy8vIFNLSVA6IERvZXNuJ3QgcnVuIHRoZSB0ZXN0LgorLy8gU0xPVzogVGhlIHRlc3QgdGFr ZXMgYSBsb25nIHRpbWUgdG8gcnVuLCBidXQgZG9lcyBub3QgdGltZW91dCBpbmRlZmluaXRlbHku CisvLyBERUZFUjogRm9yIHRlc3RzIHdlJ2xsIGZpeCBpbiB0aGUgZnV0dXJlLgorLy8gV09OVEZJ WDogRm9yIHRlc3RzIHRoYXQgd2UgbmV2ZXIgaW50ZW5kIHRvIHBhc3Mgb24gYSBnaXZlbiBwbGF0 Zm9ybS4KKy8vIERFQlVHOiBFeHBlY3RhdGlvbnMgYXBwbHkgb25seSB0byB0aGUgZGVidWcgYnVp bGQuCisvLyBSRUxFQVNFOiBFeHBlY3RhdGlvbnMgYXBwbHkgb25seSB0byByZWxlYXNlIGJ1aWxk LgorLy8gTUFDL01BQy1MRU9QQVJEL01BQy1USUdFUi9NQUMtU05PV0xFT1BBUkQ6IEV4cGVjdGF0 aW9ucyBhcHBseSBvbmx5IHRvCisvLyAgICAgdGhlc2UgcGxhdGZvcm1zLgorLy8KKy8vIFZhbGlk IHZhbHVlcyBmb3IgRVhQRUNUQVRJT05TOgorLy8KKy8vIFBBU1M6IFRlc3Qgc2hvdWxkIHBhc3MK Ky8vIENSQVNIOiBUZXN0IHNob3VsZCBjcmFzaAorLy8gVElNRU9VVDogVGVzdCBzaG91bGQgdGlt ZW91dAorLy8gSU1BR0U6IFRlc3Qgd2lsbCBwcm9kdWNlIGFuIGluY29ycmVjdCBpbWFnZSBkdW1w CisvLyBURVhUOiBUZXN0IHdpbGwgcHJvZHVjZSBpbmNvcnJlY3QgdGV4dCBvdXRwdXQgKGVpdGhl ciBmdWxsIG9yIHNpbXBsaWZpZWQpCisvLyBJTUFHRStURVhUOiBUZXN0IHdpbGwgcHJvZHVjZSBi b3RoIGluY29ycmVjdCB0ZXh0IGFuZCBpbWFnZXMKKy8vIEZBSUw6IFRlc3Qgd2lsbCBwcm9kdWNl IGVpdGhlciBpbmNvcnJlY3QgdGV4dCwgaW1hZ2VzLCBvciBib3RoCisvLworLy8gSWYgYSBsaW5l IGNvbnRhaW5zIG11bHRpcGxlIGV4cGVjdGF0aW9ucywgaXQgaXMgY29uc2lkZXJlZCAiZmxha2V5 IiwKKy8vIGJlY2F1c2UgaXQgbWVhbnMgdGhhdCB3ZSdyZSBub3QgcHJvZHVjaW5nIGNvbnNpc3Rl bnQsIHJlbGlhYmxlIHJlc3VsdHMuCisvLworLy8gRm9yIGV4YW1wbGUsCisvLyBNQUMgOiBtZWRp YSA9IFBBU1MgRkFJTAorLy8KKy8vIFRoZSBhYm92ZSBtZWFucyB0aGF0IGFsbCB0aGUgbWVkaWEg dGVzdHMgYXJlIGZsYWt5LCBidXQgb25seSBvbiBMaW51eC4KKy8vCisvLyBPdGhlciBleGFtcGxl czoKKy8vCisvLyBTS0lQIDogZmFzdC9qcy9uby1nb29kLmpzID0gVElNRU9VVCBQQVNTCisvLyBE RUJVRyA6IGZhc3QvanMvbm8tZ29vZC5qcyA9IFRJTUVPVVQgUEFTUworLy8gREVCVUcgU0tJUCA6 IGZhc3QvanMvbm8tZ29vZC5qcyA9IFRJTUVPVVQgUEFTUworLy8gTElOVVggREVCVUcgU0tJUCA6 IGZhc3QvanMvbm8tZ29vZC5qcyA9IFRJTUVPVVQgUEFTUworLy8gQlVHMTIzNCBERUJVRyBNQUMg OiBmYXN0L2pzL25vLWdvb2QuanMgPSBUSU1FT1VUIFBBU1MKKy8vCisvLyBOb3RlczoKKy8vIC1B IHRlc3QgY2Fubm90IGJlIGJvdGggU0xPVyBhbmQgVElNRU9VVAorLy8gLUEgdGVzdCBjYW4gYmUg aW5jbHVkZWQgdHdpY2UsIGJ1dCBub3QgdmlhIHRoZSBzYW1lIHBhdGguCisvLyAtSWYgYSB0ZXN0 IGlzIGluY2x1ZGVkIHR3aWNlLCB0aGVuIHRoZSBtb3JlIHByZWNpc2UgcGF0aCB3aW5zLgorLy8g LUNSQVNIIHRlc3RzIGNhbm5vdCBiZSBXT05URklYCmRpZmYgLS1naXQgYS9XZWJLaXRUb29scy9D aGFuZ2VMb2cgYi9XZWJLaXRUb29scy9DaGFuZ2VMb2cKaW5kZXggYWVmOTM4OS4uN2IwMDg5ZiAx MDA2NDQKLS0tIGEvV2ViS2l0VG9vbHMvQ2hhbmdlTG9nCisrKyBiL1dlYktpdFRvb2xzL0NoYW5n ZUxvZwpAQCAtMiw2ICsyLDE4IEBACiAKICAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BT ISkuCiAKKyAgICAgICAgQ2hhbmdlIHRoZSBNYWMgcG9ydCBvZiBuZXctcnVuLXdlYmtpdC10ZXN0 cyB0byBsb29rIGZvciBhCisgICAgICAgIHRlc3RfZXhwZWN0YXRpb25zLnR4dCBmaWxlIGluIGFk ZGl0aW9uIHRvIHRoZSBTa2lwcGVkIGZpbGVzLCBzbyB3ZQorICAgICAgICBjYW4gdHJhY2sgcGl4 ZWwgZmFpbHVyZXMuCisKKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcu Y2dpP2lkPTM2NjE5CisKKyAgICAgICAgKiBTY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9w b3J0L21hYy5weToKKworMjAxMC0wMy0yNSAgRGlyayBQcmFua2UgIDxkcHJhbmtlQGNocm9taXVt Lm9yZz4KKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKwogICAgICAgICBJ bXBsZW1lbnQgcGl4ZWwgdGVzdHMgKGltYWdlIGRpZmYpIHByb3Blcmx5IG9uIHRoZSBNYWMgcG9y dC4gQWxzbwogICAgICAgICByZWZvcm1hdCB0byBQRVAgOCBzdHlsZSBndWlkZWxpbmVzLgogCmRp ZmYgLS1naXQgYS9XZWJLaXRUb29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9wb3J0 L21hYy5weSBiL1dlYktpdFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3BvcnQv bWFjLnB5CmluZGV4IDliZDJjZDMuLmIxYjU3NWMgMTAwNjQ0Ci0tLSBhL1dlYktpdFRvb2xzL1Nj cmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3BvcnQvbWFjLnB5CisrKyBiL1dlYktpdFRvb2xz L1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3BvcnQvbWFjLnB5CkBAIC0xNTAsNiArMTUw LDEwIEBAIGNsYXNzIE1hY1BvcnQoYmFzZS5Qb3J0KToKICAgICBkZWYgbnVtX2NvcmVzKHNlbGYp OgogICAgICAgICByZXR1cm4gaW50KG9zLnBvcGVuMigic3lzY3RsIC1uIGh3Lm5jcHUiKVsxXS5y ZWFkKCkpCiAKKyAgICBkZWYgcGF0aF90b190ZXN0X2V4cGVjdGF0aW9uc19maWxlKHNlbGYpOgor ICAgICAgICByZXR1cm4gc2VsZi5wYXRoX2Zyb21fd2Via2l0X2Jhc2UoJ0xheW91dFRlc3RzJywg J3BsYXRmb3JtJywKKyAgICAgICAgICAgICdtYWMnLCAndGVzdF9leHBlY3RhdGlvbnMudHh0JykK KyAKICAgICBkZWYgcmVzdWx0c19kaXJlY3Rvcnkoc2VsZik6CiAgICAgICAgIHJldHVybiAoJy90 bXAvcnVuLWNocm9taXVtLXdlYmtpdC10ZXN0cy0nICsKICAgICAgICAgICAgICAgICBzZWxmLl9v cHRpb25zLnJlc3VsdHNfZGlyZWN0b3J5KQpAQCAtMjQwLDMyICsyNDQsMzUgQEAgY2xhc3MgTWFj UG9ydChiYXNlLlBvcnQpOgogICAgICAgICByZXR1cm4gdGVzdHNfdG9fc2tpcAogCiAgICAgZGVm IHRlc3RfZXhwZWN0YXRpb25zKHNlbGYpOgotICAgICAgICAjIFRoZSBXZWJLaXQgbWFjIHBvcnQg dXNlcyAnU2tpcHBlZCcgZmlsZXMgYXQgdGhlIG1vbWVudC4gRWFjaAotICAgICAgICAjIGZpbGUg Y29udGFpbnMgYSBsaXN0IG9mIGZpbGVzIG9yIGRpcmVjdG9yaWVzIHRvIGJlIHNraXBwZWQgZHVy aW5nCi0gICAgICAgICMgdGhlIHRlc3QgcnVuLiBUaGUgdG90YWwgbGlzdCBvZiB0ZXN0cyB0byBz a2lwcGVkIGlzIGdpdmVuIGJ5IHRoZQotICAgICAgICAjIGNvbnRlbnRzIG9mIHRoZSBnZW5lcmlj IFNraXBwZWQgZmlsZSBmb3VuZCBpbiBwbGF0Zm9ybS9YIHBsdXMKLSAgICAgICAgIyBhIHZlcnNp b24tc3BlY2lmaWMgZmlsZSBmb3VuZCBpbiBwbGF0Zm9ybS9YLXZlcnNpb24uIER1cGxpY2F0ZQot ICAgICAgICAjIGVudHJpZXMgYXJlIGFsbG93ZWQuIFRoaXMgcm91dGluZSByZWFkcyB0aG9zZSBm aWxlcyBhbmQgdHVybnMKLSAgICAgICAgIyBjb250ZW50cyBpbnRvIHRoZSBmb3JtYXQgZXhwZWN0 ZWQgYnkgdGVzdF9leHBlY3RhdGlvbnMuCisgICAgICAgICMgVGhlIFdlYktpdCBtYWMgcG9ydCB1 c2VzIGEgY29tYmluYXRpb24gb2YgYSB0ZXN0X2V4cGVjdGF0aW9ucyBmaWxlCisgICAgICAgICMg YW5kICdTa2lwcGVkJyBmaWxlcy4KKyAgICAgICAgZXhwZWN0YXRpb25zX2ZpbGUgPSBzZWxmLnBh dGhfdG9fdGVzdF9leHBlY3RhdGlvbnNfZmlsZSgpCisgICAgICAgIGV4cGVjdGF0aW9ucyA9IGZp bGUoZXhwZWN0YXRpb25zX2ZpbGUsICJyIikucmVhZCgpCisgICAgICAgIHJldHVybiBleHBlY3Rh dGlvbnMgKyBzZWxmLl9za2lwcygpCisKKyAgICBkZWYgX3NraXBzKHNlbGYpOgorICAgICAgICAj IEVhY2ggU2tpcHBlZCBmaWxlIGNvbnRhaW5zIGEgbGlzdCBvZiBmaWxlcworICAgICAgICAjIG9y IGRpcmVjdG9yaWVzIHRvIGJlIHNraXBwZWQgZHVyaW5nIHRoZSB0ZXN0IHJ1bi4gVGhlIHRvdGFs IGxpc3QKKyAgICAgICAgIyBvZiB0ZXN0cyB0byBza2lwcGVkIGlzIGdpdmVuIGJ5IHRoZSBjb250 ZW50cyBvZiB0aGUgZ2VuZXJpYworICAgICAgICAjIFNraXBwZWQgZmlsZSBmb3VuZCBpbiBwbGF0 Zm9ybS9YIHBsdXMgYSB2ZXJzaW9uLXNwZWNpZmljIGZpbGUKKyAgICAgICAgIyBmb3VuZCBpbiBw bGF0Zm9ybS9YLXZlcnNpb24uIER1cGxpY2F0ZSBlbnRyaWVzIGFyZSBhbGxvd2VkLgorICAgICAg ICAjIFRoaXMgcm91dGluZSByZWFkcyB0aG9zZSBmaWxlcyBhbmQgdHVybnMgY29udGVudHMgaW50 byB0aGUKKyAgICAgICAgIyBmb3JtYXQgZXhwZWN0ZWQgYnkgdGVzdF9leHBlY3RhdGlvbnMuCiAK ICAgICAgICAgIyBVc2UgYSBzZXQgdG8gYWxsb3cgZHVwbGljYXRlcwogICAgICAgICB0ZXN0c190 b19za2lwID0gc2V0KHNlbGYuX2V4cGVjdGF0aW9uc19mcm9tX3NraXBwZWRfZmlsZXMoKSkKIAog ICAgICAgICB0ZXN0c190b19za2lwLnVwZGF0ZShzZWxmLl90ZXN0c19mb3Jfb3RoZXJfcGxhdGZv cm1zKCkpCiAgICAgICAgIHRlc3RzX3RvX3NraXAudXBkYXRlKHNlbGYuX3Rlc3RzX2Zvcl9kaXNh YmxlZF9mZWF0dXJlcygpKQotICAgICAgICBleHBlY3RhdGlvbnMgPSBtYXAobGFtYmRhIHRlc3Rf cGF0aDogIkJVR19TS0lQUEVEIFNLSVAgOiAlcyA9IEZBSUwiICUKLSAgICAgICAgICAgICAgICAg ICAgICAgICAgIHRlc3RfcGF0aCwgdGVzdHNfdG9fc2tpcCkKLSAgICAgICAgcmV0dXJuICJcbiIu am9pbihleHBlY3RhdGlvbnMpCisgICAgICAgIHNraXBfbGluZXMgPSBtYXAobGFtYmRhIHRlc3Rf cGF0aDogIkJVR19TS0lQUEVEIFNLSVAgOiAlcyA9IEZBSUwiICUKKyAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgdGVzdF9wYXRoLCB0ZXN0c190b19za2lwKQorICAgICAgICByZXR1cm4g IlxuIi5qb2luKHNraXBfbGluZXMpCiAKICAgICBkZWYgdGVzdF9wbGF0Zm9ybV9uYW1lKHNlbGYp OgotICAgICAgICAjIEF0IHRoZSBtb21lbnQgd2UgZG9uJ3QgdXNlIHRlc3QgcGxhdGZvcm0gbmFt ZXMsIGJ1dCB3ZSBoYXZlCi0gICAgICAgICMgdG8gcmV0dXJuIHNvbWV0aGluZy4KLSAgICAgICAg cmV0dXJuICdtYWMnCisgICAgICAgIHJldHVybiAnbWFjJyArIHNlbGYudmVyc2lvbigpCiAKICAg ICBkZWYgdGVzdF9wbGF0Zm9ybV9uYW1lcyhzZWxmKToKLSAgICAgICAgIyBBdCB0aGUgbW9tZW50 IHdlIGRvbid0IHVzZSB0ZXN0IHBsYXRmb3JtIG5hbWVzLCBidXQgd2UgaGF2ZQotICAgICAgICAj IHRvIHJldHVybiBzb21ldGhpbmcuCi0gICAgICAgIHJldHVybiAoJ21hYycsKQorICAgICAgICBy ZXR1cm4gKCdtYWMnLCAnbWFjLXRpZ2VyJywgJ21hYy1sZW9wYXJkJywgJ21hYy1zbm93bGVvcGFy ZCcpCiAKICAgICBkZWYgdmVyc2lvbihzZWxmKToKICAgICAgICAgb3NfdmVyc2lvbl9zdHJpbmcg PSBwbGF0Zm9ybS5tYWNfdmVyKClbMF0gICMgZS5nLiAiMTAuNS42Igo=