105583 2012-12-20 15:10:13 -0800 TestExpectation fallback is broken 2013-01-30 14:45:11 -0800 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified RESOLVED FIXED P2 Normal --- 1 simon.fraser dpranke dpranke jonlee ojan rniwa simon.fraser oldest_to_newest 795850 0 simon.fraser 2012-12-20 15:10:13 -0800 http/tests/notifications is skipped in platform/mac-lion/TestExpectations but doesn't occur in any other TestExpectaions file. On Mac Lion, the fallback path shows as: 13:48:20.439 89663 Test configuration: <lion, x86_64, release> 13:48:20.439 89663 Placing test results in /Volumes/Data/slave/lion-release-tests-wk2/build/layout-test-results 13:48:20.439 89663 Baseline search path: mac-wk2 -> mac-lion -> mac -> generic but these tests are still running. webkitpy is so opaque that I wasn't able to figure out why. 795852 1 simon.fraser 2012-12-20 15:11:08 -0800 An example of these tests getting run: http://build.webkit.org/builders/Apple%20Lion%20Release%20WK2%20%28Tests%29/builds/6513/steps/layout-test/logs/stdio 795865 2 simon.fraser 2012-12-20 15:29:39 -0800 So on Lion, running r-w-t -2 parses: LayoutTests/platform/mac/TestExpectations LayoutTests/platform/mac-wk2/TestExpectations LayoutTests/platform/wk2/TestExpectations Running without -2 parses: LayoutTests/platform/mac/TestExpectations LayoutTests/platform/mac-lion/TestExpectations This is clearly broken. 795870 3 dpranke 2012-12-20 15:36:51 -0800 argh. the whole mac-lion file is being skipped on the -wk2 port. looks like I broke this when I added the support for the -wk2 port names. fixing now ... 795887 4 rniwa 2012-12-20 15:51:15 -0800 We can just use the default baseline fallback paths for WK2 ports. There is no point in restricting search paths for TestExpectations here. 795893 5 180430 dpranke 2012-12-20 15:56:28 -0800 Created attachment 180430 Patch 795896 6 180430 rniwa 2012-12-20 15:57:49 -0800 Comment on attachment 180430 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=180430&action=review > Tools/ChangeLog:15 > + * Scripts/webkitpy/layout_tests/port/port_testcase.py: No new tests? 795901 7 dpranke 2012-12-20 16:00:37 -0800 (In reply to comment #4) > We can just use the default baseline fallback paths for WK2 ports. There is no point in restricting search paths for TestExpectations here. I'm not sure I follow you; are you suggesting we should use the same fallback path for expectations files that we do for baselines? (In reply to comment #6) > (From update of attachment 180430 [details]) > View in context: https://bugs.webkit.org/attachment.cgi?id=180430&action=review > > > Tools/ChangeLog:15 > > + * Scripts/webkitpy/layout_tests/port/port_testcase.py: > > No new tests? There were existing tests that attempted to cover this case, but they were doing it wrong (manipulating internal flags like self._name rather than using the public APIs). I changed the existing tests to parse the port names properly and that catches the problem. 795902 8 dpranke 2012-12-20 16:02:33 -0800 (In reply to comment #7) > (In reply to comment #4) > > We can just use the default baseline fallback paths for WK2 ports. There is no point in restricting search paths for TestExpectations here. > > I'm not sure I follow you; are you suggesting we should use the same fallback path for expectations files that we do for baselines? > Historically we used a different list for Skipped files than we did for baselines, that was because there was no way to "un-skip" a file (so if you skipped a file in platform/mac, it would also be skipped regardless of what is in platform/mac-lion). You can un-skip things in TestExpectation, so your suggestion would be fine. I don't have a strong opinion here (although I think consistency would be good so would probably agree that that's the right thing to do), but it's up to the folks who use multiple files to chime in. 795903 9 rniwa 2012-12-20 16:03:05 -0800 (In reply to comment #7) > (In reply to comment #4) > > We can just use the default baseline fallback paths for WK2 ports. There is no point in restricting search paths for TestExpectations here. > > I'm not sure I follow you; are you suggesting we should use the same fallback path for expectations files that we do for baselines? Yes. I think we're adding too much complexity here. > (In reply to comment #6) > > No new tests? > > There were existing tests that attempted to cover this case, but they were doing it wrong (manipulating internal flags like self._name rather than using the public APIs). I changed the existing tests to parse the port names properly and that catches the problem. Okay. Thanks for the clarification. 795904 10 dpranke 2012-12-20 16:05:19 -0800 (In reply to comment #9) > (In reply to comment #7) > > (In reply to comment #4) > > > We can just use the default baseline fallback paths for WK2 ports. There is no point in restricting search paths for TestExpectations here. > > > > I'm not sure I follow you; are you suggesting we should use the same fallback path for expectations files that we do for baselines? > > Yes. I think we're adding too much complexity here. > As I said, fine by me. I'll post a separate patch for that and affected folks can chime in if they do or don't want the change. I'll land this patch now to fix the tree, though ... 795906 11 180430 dpranke 2012-12-20 16:05:53 -0800 Comment on attachment 180430 Patch Clearing flags on attachment: 180430 Committed r138314: <http://trac.webkit.org/changeset/138314> 795907 12 dpranke 2012-12-20 16:05:56 -0800 All reviewed patches have been landed. Closing bug. 795934 13 simon.fraser 2012-12-20 16:26:44 -0800 It's also distressing that the "Baseline search path: mac-wk2 -> mac-lion -> mac -> generic" output was incorrect, and didn't reflect which expectations files were actually being parsed. Is that fixed with the patch? 795943 14 rniwa 2012-12-20 16:35:08 -0800 (In reply to comment #13) > It's also distressing that the "Baseline search path: mac-wk2 -> mac-lion -> mac -> generic" output was incorrect, and didn't reflect which expectations files were actually being parsed. Is that fixed with the patch? That's expected. The baseline search path was correct. The underlying problem here is that the baselinse search paths is DIFFERENT from search paths for TestExpectations files, which is mind blowingly counter intuitive to say the least. I think we should change it to match the baseline search paths. 795968 15 dpranke 2012-12-20 17:01:36 -0800 (In reply to comment #14) > (In reply to comment #13) > > It's also distressing that the "Baseline search path: mac-wk2 -> mac-lion -> mac -> generic" output was incorrect, and didn't reflect which expectations files were actually being parsed. Is that fixed with the patch? > > That's expected. The baseline search path was correct. The underlying problem here is that the baselinse search paths is DIFFERENT from search paths for TestExpectations files, which is mind blowingly counter intuitive to say the least. > > I think we should change it to match the baseline search paths. Right. 820602 16 simon.fraser 2013-01-30 14:41:37 -0800 (In reply to comment #15) > (In reply to comment #14) > > (In reply to comment #13) > > > It's also distressing that the "Baseline search path: mac-wk2 -> mac-lion -> mac -> generic" output was incorrect, and didn't reflect which expectations files were actually being parsed. Is that fixed with the patch? > > > > That's expected. The baseline search path was correct. The underlying problem here is that the baselinse search paths is DIFFERENT from search paths for TestExpectations files, which is mind blowingly counter intuitive to say the least. > > > > I think we should change it to match the baseline search paths. > > Right. Is there a bug to track that? 820612 17 rniwa 2013-01-30 14:45:11 -0800 (In reply to comment #16) > (In reply to comment #15) > > (In reply to comment #14) > > > (In reply to comment #13) > > > > It's also distressing that the "Baseline search path: mac-wk2 -> mac-lion -> mac -> generic" output was incorrect, and didn't reflect which expectations files were actually being parsed. Is that fixed with the patch? > > > > > > That's expected. The baseline search path was correct. The underlying problem here is that the baselinse search paths is DIFFERENT from search paths for TestExpectations files, which is mind blowingly counter intuitive to say the least. > > > > > > I think we should change it to match the baseline search paths. > > > > Right. > > Is there a bug to track that? Has been fixed in https://bugs.webkit.org/show_bug.cgi?id=105599. 180430 2012-12-20 15:56:28 -0800 2012-12-20 16:05:53 -0800 Patch bug-105583-20121220155344.patch text/plain 8281 dpranke U3VidmVyc2lvbiBSZXZpc2lvbjogMTM4MjkxCmRpZmYgLS1naXQgYS9Ub29scy9DaGFuZ2VMb2cg Yi9Ub29scy9DaGFuZ2VMb2cKaW5kZXggYjc0MmUxNzBjNzgxNDVkNDBiYWMyNTAwNjhhMDBkYzI0 OTljMWQ3Yy4uYTNhYTk1MTdlMGM1ZDEyYzYwYzIyYzQ3NTg4MDQwZThkZDYxMGEwMiAxMDA2NDQK LS0tIGEvVG9vbHMvQ2hhbmdlTG9nCisrKyBiL1Rvb2xzL0NoYW5nZUxvZwpAQCAtMSwzICsxLDI2 IEBACisyMDEyLTEyLTIwICBEaXJrIFByYW5rZSAgPGRwcmFua2VAY2hyb21pdW0ub3JnPgorCisg ICAgICAgIFRlc3RFeHBlY3RhdGlvbiBmYWxsYmFjayBpcyBicm9rZW4KKyAgICAgICAgaHR0cHM6 Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTEwNTU4MworCisgICAgICAgIFJldmll d2VkIGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgIFdoZW4gSSBhZGRlZCBzdXBwb3J0IGZv ciB0aGUgLXdrMiBpbiBwb3J0IG5hbWVzLCBpdCBsb29rcworICAgICAgICBsaWtlIEkgYnJva2Ug dGhlIHdheSB3ZSBjb21wdXRlZCB0aGUgbGlzdCBvZiBUZXN0RXhwZWN0YXRpb25zCisgICAgICAg IGZpbGVzIHNvIHRoYXQgdGhlIGltcGxlbWVudGF0aW9uLXZlcnNpb24gZGlyZWN0b3J5IHdhcyBs ZWZ0IG91dCA6KC4KKyAgICAgICAgRml4ZWQgYW5kIGNsZWFuZWQgdXAgdGhlIHRlc3RzLgorCisg ICAgICAgICogU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvcG9ydC9iYXNlLnB5OgorICAg ICAgICAoUG9ydC5leHBlY3RhdGlvbnNfZmlsZXMpOgorICAgICAgICAqIFNjcmlwdHMvd2Via2l0 cHkvbGF5b3V0X3Rlc3RzL3BvcnQvcG9ydF90ZXN0Y2FzZS5weToKKyAgICAgICAgKFRlc3RXZWJL aXRQb3J0Ll9faW5pdF9fKToKKyAgICAgICAgKFBvcnRUZXN0Q2FzZS50ZXN0X3NraXBwZWRfZGly ZWN0b3JpZXNfZm9yX3N5bWJvbHMpOgorICAgICAgICAodGVzdF9za2lwcGVkX2RpcmVjdG9yaWVz X2Zvcl9mZWF0dXJlcyk6CisgICAgICAgICh0ZXN0X3NraXBwZWRfZGlyZWN0b3JpZXNfZm9yX2Zl YXR1cmVzX25vX21hdGNoaW5nX3Rlc3RzX2luX3Rlc3RfbGlzdCk6CisgICAgICAgICh0ZXN0X3Nr aXBwZWRfdGVzdHNfZm9yX3Vuc3VwcG9ydGVkX2ZlYXR1cmVzX2VtcHR5X3Rlc3RfbGlzdCk6Cisg ICAgICAgICh0ZXN0X3NraXBwZWRfbGF5b3V0X3Rlc3RzKToKKyAgICAgICAgKHRlc3RfZXhwZWN0 YXRpb25zX2ZpbGVzKToKKwogMjAxMi0xMi0xOSAgU2ltb24gRnJhc2VyICA8c2ltb24uZnJhc2Vy QGFwcGxlLmNvbT4KIAogICAgICAgICBMb3RzIG9mIHN0aWNreSB0ZXN0cyBmYWlsaW5nIGluIFdL MgpkaWZmIC0tZ2l0IGEvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvcG9ydC9i YXNlLnB5IGIvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvcG9ydC9iYXNlLnB5 CmluZGV4IDkzYjk2NDVlMzBiNDIyNWQ4OTNkOTAyNGZhMWFiNDYzOWQ3MTIxM2QuLjhlZDA1NzA4 ZjQ2MTY1YWFkNjUxMGUyYTNlODI3Zjg5MmY5OGMyZGMgMTAwNjQ0Ci0tLSBhL1Rvb2xzL1Njcmlw dHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3BvcnQvYmFzZS5weQorKysgYi9Ub29scy9TY3JpcHRz L3dlYmtpdHB5L2xheW91dF90ZXN0cy9wb3J0L2Jhc2UucHkKQEAgLTEwNzEsMTMgKzEwNzEsMTUg QEAgY2xhc3MgUG9ydChvYmplY3QpOgogICAgICAgICAjIFVubGlrZSBiYXNlbGluZV9zZWFyY2hf cGF0aCwgd2Ugb25seSB3YW50IHRvIHNlYXJjaCBbV0syLVBPUlQsIFBPUlQtVkVSU0lPTiwgUE9S VF0gYW5kIGFueSBkaXJlY3RvcmllcwogICAgICAgICAjIGluY2x1ZGVkIHZpYSAtLWFkZGl0aW9u YWwtcGxhdGZvcm0tZGlyZWN0b3J5LCBub3QgdGhlIGZ1bGwgY2FzYWRlLgogICAgICAgICBzZWFy Y2hfcGF0aHMgPSBbc2VsZi5wb3J0X25hbWVdCi0gICAgICAgIGlmIHNlbGYubmFtZSgpICE9IHNl bGYucG9ydF9uYW1lOgotICAgICAgICAgICAgc2VhcmNoX3BhdGhzLmFwcGVuZChzZWxmLm5hbWUo KSkKKworICAgICAgICBub25fd2syX25hbWUgPSBzZWxmLm5hbWUoKS5yZXBsYWNlKCctd2syJywg JycpCisgICAgICAgIGlmIG5vbl93azJfbmFtZSAhPSBzZWxmLnBvcnRfbmFtZToKKyAgICAgICAg ICAgIHNlYXJjaF9wYXRocy5hcHBlbmQobm9uX3drMl9uYW1lKQogCiAgICAgICAgIGlmIHNlbGYu Z2V0X29wdGlvbignd2Via2l0X3Rlc3RfcnVubmVyJyk6CiAgICAgICAgICAgICAjIEJlY2F1c2Ug bmVhcmx5IGFsbCBvZiB0aGUgc2tpcHBlZCB0ZXN0cyBmb3IgV2ViS2l0IDIgYXJlIGR1ZSB0byBj cm9zcy1wbGF0Zm9ybQogICAgICAgICAgICAgIyBpc3N1ZXMsIGFsbCB3azIgcG9ydHMgc2hhcmUg YSBza2lwcGVkIGxpc3QgdW5kZXIgcGxhdGZvcm0vd2syLgotICAgICAgICAgICAgc2VhcmNoX3Bh dGhzLmV4dGVuZChbc2VsZi5fd2syX3BvcnRfbmFtZSgpLCAid2syIl0pCisgICAgICAgICAgICBz ZWFyY2hfcGF0aHMuZXh0ZW5kKFsid2syIiwgc2VsZi5fd2syX3BvcnRfbmFtZSgpXSkKIAogICAg ICAgICBzZWFyY2hfcGF0aHMuZXh0ZW5kKHNlbGYuZ2V0X29wdGlvbigiYWRkaXRpb25hbF9wbGF0 Zm9ybV9kaXJlY3RvcnkiLCBbXSkpCiAKZGlmZiAtLWdpdCBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0 cHkvbGF5b3V0X3Rlc3RzL3BvcnQvcG9ydF90ZXN0Y2FzZS5weSBiL1Rvb2xzL1NjcmlwdHMvd2Vi a2l0cHkvbGF5b3V0X3Rlc3RzL3BvcnQvcG9ydF90ZXN0Y2FzZS5weQppbmRleCA2YzJlY2YxMGI3 OTk1NWQ1YmIzNDJlYWU4YjZiZjAzMGU5ODJlOTZhLi42NmZiNTkwNWJmMjM2YjM1OWVmYTNhZDky Mjg3YjRkOTBmMmQ0MTk4IDEwMDY0NAotLS0gYS9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91 dF90ZXN0cy9wb3J0L3BvcnRfdGVzdGNhc2UucHkKKysrIGIvVG9vbHMvU2NyaXB0cy93ZWJraXRw eS9sYXlvdXRfdGVzdHMvcG9ydC9wb3J0X3Rlc3RjYXNlLnB5CkBAIC01MCwxMiArNTAsMTMgQEAg ZnJvbSB3ZWJraXRweS50b29sLm1vY2t0b29sIGltcG9ydCBNb2NrT3B0aW9ucwogY2xhc3MgVGVz dFdlYktpdFBvcnQoUG9ydCk6CiAgICAgcG9ydF9uYW1lID0gInRlc3R3ZWJraXRwb3J0IgogCi0g ICAgZGVmIF9faW5pdF9fKHNlbGYsIHN5bWJvbHNfc3RyaW5nPU5vbmUsCisgICAgZGVmIF9faW5p dF9fKHNlbGYsIHBvcnRfbmFtZT1Ob25lLCBzeW1ib2xzX3N0cmluZz1Ob25lLAogICAgICAgICAg ICAgICAgICBleHBlY3RhdGlvbnNfZmlsZT1Ob25lLCBza2lwc19maWxlPU5vbmUsIGhvc3Q9Tm9u ZSwgY29uZmlnPU5vbmUsCiAgICAgICAgICAgICAgICAgICoqa3dhcmdzKToKKyAgICAgICAgcG9y dF9uYW1lID0gcG9ydF9uYW1lIG9yIFRlc3RXZWJLaXRQb3J0LnBvcnRfbmFtZQogICAgICAgICBz ZWxmLnN5bWJvbHNfc3RyaW5nID0gc3ltYm9sc19zdHJpbmcgICMgUGFzc2luZyAiIiBkaXNhYmxl cyBhbGwgc3RhdGljbHktZGV0ZWN0YWJsZSBmZWF0dXJlcy4KICAgICAgICAgaG9zdCA9IGhvc3Qg b3IgTW9ja1N5c3RlbUhvc3QoKQotICAgICAgICBzdXBlcihUZXN0V2ViS2l0UG9ydCwgc2VsZiku X19pbml0X18oaG9zdCwgVGVzdFdlYktpdFBvcnQucG9ydF9uYW1lLCAqKmt3YXJncykKKyAgICAg ICAgc3VwZXIoVGVzdFdlYktpdFBvcnQsIHNlbGYpLl9faW5pdF9fKGhvc3QsIHBvcnRfbmFtZT1w b3J0X25hbWUsICoqa3dhcmdzKQogCiAgICAgZGVmIGFsbF90ZXN0X2NvbmZpZ3VyYXRpb25zKHNl bGYpOgogICAgICAgICByZXR1cm4gW3NlbGYudGVzdF9jb25maWd1cmF0aW9uKCldCkBAIC00NjIs NyArNDYzLDcgQEAgY2xhc3MgUG9ydFRlc3RDYXNlKHVuaXR0ZXN0LlRlc3RDYXNlKToKICAgICAg ICAgICAgICJpbnNwZWN0b3Ivc3R5bGVzL3ZhcmlhYmxlcyIsICAjIFJlcXVpcmVzIENTUyBWYXJp YWJsZXMKICAgICAgICAgXSkKIAotICAgICAgICByZXN1bHRfZGlyZWN0b3JpZXMgPSBzZXQoVGVz dFdlYktpdFBvcnQoc3ltYm9sc19zdHJpbmcsIE5vbmUpLl9za2lwcGVkX3Rlc3RzX2Zvcl91bnN1 cHBvcnRlZF9mZWF0dXJlcyh0ZXN0X2xpc3Q9WydtYXRobWwvZm9vLmh0bWwnXSkpCisgICAgICAg IHJlc3VsdF9kaXJlY3RvcmllcyA9IHNldChUZXN0V2ViS2l0UG9ydChzeW1ib2xzX3N0cmluZz1z eW1ib2xzX3N0cmluZykuX3NraXBwZWRfdGVzdHNfZm9yX3Vuc3VwcG9ydGVkX2ZlYXR1cmVzKHRl c3RfbGlzdD1bJ21hdGhtbC9mb28uaHRtbCddKSkKICAgICAgICAgc2VsZi5hc3NlcnRFcXVhbChy ZXN1bHRfZGlyZWN0b3JpZXMsIGV4cGVjdGVkX2RpcmVjdG9yaWVzKQogCiAgICAgICAgICMgVGVz dCB0aGF0IHRoZSBubSBzdHJpbmcgcGFyc2luZyBhY3R1YWxseSB3b3JrczoKQEAgLTQ3MywxMyAr NDc0LDEzIEBAIGNsYXNzIFBvcnRUZXN0Q2FzZSh1bml0dGVzdC5UZXN0Q2FzZSk6CiAiIiIKICAg ICAgICAgIyBOb3RlICdjb21wb3NpdGluZycgaXMgbm90IGluIHRoZSBsaXN0IG9mIHNraXBwZWQg ZGlyZWN0b3JpZXMgKGhlbmNlIHRoZSBwYXJzaW5nIG9mIEdyYXBoaWNzTGF5ZXIgd29ya2VkKToK ICAgICAgICAgZXhwZWN0ZWRfZGlyZWN0b3JpZXMgPSBzZXQoWydtYXRobWwnLCAndHJhbnNmb3Jt cy8zZCcsICdjb21wb3NpdGluZy93ZWJnbCcsICdmYXN0L2NhbnZhcy93ZWJnbCcsICdhbmltYXRp b25zLzNkJywgJ21odG1sJywgJ2h0dHAvdGVzdHMvY2FudmFzL3dlYmdsJywgJ2Zhc3QvY3NzL3Zh cmlhYmxlcycsICdpbnNwZWN0b3Ivc3R5bGVzL3ZhcmlhYmxlcyddKQotICAgICAgICByZXN1bHRf ZGlyZWN0b3JpZXMgPSBzZXQoVGVzdFdlYktpdFBvcnQoc3ltYm9sc19zdHJpbmcsIE5vbmUpLl9z a2lwcGVkX3Rlc3RzX2Zvcl91bnN1cHBvcnRlZF9mZWF0dXJlcyh0ZXN0X2xpc3Q9WydtYXRobWwv Zm9vLmh0bWwnXSkpCisgICAgICAgIHJlc3VsdF9kaXJlY3RvcmllcyA9IHNldChUZXN0V2ViS2l0 UG9ydChzeW1ib2xzX3N0cmluZz1zeW1ib2xzX3N0cmluZykuX3NraXBwZWRfdGVzdHNfZm9yX3Vu c3VwcG9ydGVkX2ZlYXR1cmVzKHRlc3RfbGlzdD1bJ21hdGhtbC9mb28uaHRtbCddKSkKICAgICAg ICAgc2VsZi5hc3NlcnRFcXVhbChyZXN1bHRfZGlyZWN0b3JpZXMsIGV4cGVjdGVkX2RpcmVjdG9y aWVzKQogCiAgICAgZGVmIHRlc3Rfc2tpcHBlZF9kaXJlY3Rvcmllc19mb3JfZmVhdHVyZXMoc2Vs Zik6CiAgICAgICAgIHN1cHBvcnRlZF9mZWF0dXJlcyA9IFsiQWNjZWxlcmF0ZWQgQ29tcG9zaXRp bmciLCAiRm9vIEZlYXR1cmUiXQogICAgICAgICBleHBlY3RlZF9kaXJlY3RvcmllcyA9IHNldChb ImFuaW1hdGlvbnMvM2QiLCAidHJhbnNmb3Jtcy8zZCJdKQotICAgICAgICBwb3J0ID0gVGVzdFdl YktpdFBvcnQoTm9uZSwgc3VwcG9ydGVkX2ZlYXR1cmVzKQorICAgICAgICBwb3J0ID0gVGVzdFdl YktpdFBvcnQoc3VwcG9ydGVkX2ZlYXR1cmVzPXN1cHBvcnRlZF9mZWF0dXJlcykKICAgICAgICAg cG9ydC5fcnVudGltZV9mZWF0dXJlX2xpc3QgPSBsYW1iZGE6IHN1cHBvcnRlZF9mZWF0dXJlcwog ICAgICAgICByZXN1bHRfZGlyZWN0b3JpZXMgPSBzZXQocG9ydC5fc2tpcHBlZF90ZXN0c19mb3Jf dW5zdXBwb3J0ZWRfZmVhdHVyZXModGVzdF9saXN0PVsiYW5pbWF0aW9ucy8zZC9mb28uaHRtbCJd KSkKICAgICAgICAgc2VsZi5hc3NlcnRFcXVhbChyZXN1bHRfZGlyZWN0b3JpZXMsIGV4cGVjdGVk X2RpcmVjdG9yaWVzKQpAQCAtNDg3LDE3ICs0ODgsMTcgQEAgY2xhc3MgUG9ydFRlc3RDYXNlKHVu aXR0ZXN0LlRlc3RDYXNlKToKICAgICBkZWYgdGVzdF9za2lwcGVkX2RpcmVjdG9yaWVzX2Zvcl9m ZWF0dXJlc19ub19tYXRjaGluZ190ZXN0c19pbl90ZXN0X2xpc3Qoc2VsZik6CiAgICAgICAgIHN1 cHBvcnRlZF9mZWF0dXJlcyA9IFsiQWNjZWxlcmF0ZWQgQ29tcG9zaXRpbmciLCAiRm9vIEZlYXR1 cmUiXQogICAgICAgICBleHBlY3RlZF9kaXJlY3RvcmllcyA9IHNldChbXSkKLSAgICAgICAgcmVz dWx0X2RpcmVjdG9yaWVzID0gc2V0KFRlc3RXZWJLaXRQb3J0KE5vbmUsIHN1cHBvcnRlZF9mZWF0 dXJlcykuX3NraXBwZWRfdGVzdHNfZm9yX3Vuc3VwcG9ydGVkX2ZlYXR1cmVzKHRlc3RfbGlzdD1b J2Zvby5odG1sJ10pKQorICAgICAgICByZXN1bHRfZGlyZWN0b3JpZXMgPSBzZXQoVGVzdFdlYktp dFBvcnQoc3VwcG9ydGVkX2ZlYXR1cmVzPXN1cHBvcnRlZF9mZWF0dXJlcykuX3NraXBwZWRfdGVz dHNfZm9yX3Vuc3VwcG9ydGVkX2ZlYXR1cmVzKHRlc3RfbGlzdD1bJ2Zvby5odG1sJ10pKQogICAg ICAgICBzZWxmLmFzc2VydEVxdWFsKHJlc3VsdF9kaXJlY3RvcmllcywgZXhwZWN0ZWRfZGlyZWN0 b3JpZXMpCiAKICAgICBkZWYgdGVzdF9za2lwcGVkX3Rlc3RzX2Zvcl91bnN1cHBvcnRlZF9mZWF0 dXJlc19lbXB0eV90ZXN0X2xpc3Qoc2VsZik6CiAgICAgICAgIHN1cHBvcnRlZF9mZWF0dXJlcyA9 IFsiQWNjZWxlcmF0ZWQgQ29tcG9zaXRpbmciLCAiRm9vIEZlYXR1cmUiXQogICAgICAgICBleHBl Y3RlZF9kaXJlY3RvcmllcyA9IHNldChbXSkKLSAgICAgICAgcmVzdWx0X2RpcmVjdG9yaWVzID0g c2V0KFRlc3RXZWJLaXRQb3J0KE5vbmUsIHN1cHBvcnRlZF9mZWF0dXJlcykuX3NraXBwZWRfdGVz dHNfZm9yX3Vuc3VwcG9ydGVkX2ZlYXR1cmVzKHRlc3RfbGlzdD1Ob25lKSkKKyAgICAgICAgcmVz dWx0X2RpcmVjdG9yaWVzID0gc2V0KFRlc3RXZWJLaXRQb3J0KHN1cHBvcnRlZF9mZWF0dXJlcz1z dXBwb3J0ZWRfZmVhdHVyZXMpLl9za2lwcGVkX3Rlc3RzX2Zvcl91bnN1cHBvcnRlZF9mZWF0dXJl cyh0ZXN0X2xpc3Q9Tm9uZSkpCiAgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwocmVzdWx0X2RpcmVj dG9yaWVzLCBleHBlY3RlZF9kaXJlY3RvcmllcykKIAogICAgIGRlZiB0ZXN0X3NraXBwZWRfbGF5 b3V0X3Rlc3RzKHNlbGYpOgotICAgICAgICBzZWxmLmFzc2VydEVxdWFsKFRlc3RXZWJLaXRQb3J0 KE5vbmUsIE5vbmUpLnNraXBwZWRfbGF5b3V0X3Rlc3RzKHRlc3RfbGlzdD1bXSksIHNldChbJ21l ZGlhJ10pKQorICAgICAgICBzZWxmLmFzc2VydEVxdWFsKFRlc3RXZWJLaXRQb3J0KCkuc2tpcHBl ZF9sYXlvdXRfdGVzdHModGVzdF9saXN0PVtdKSwgc2V0KFsnbWVkaWEnXSkpCiAKICAgICBkZWYg dGVzdF9leHBlY3RhdGlvbnNfZmlsZXMoc2VsZik6CiAgICAgICAgIHBvcnQgPSBUZXN0V2ViS2l0 UG9ydCgpCkBAIC01MDcsMTMgKzUwOCwxNCBAQCBjbGFzcyBQb3J0VGVzdENhc2UodW5pdHRlc3Qu VGVzdENhc2UpOgogCiAgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwocGxhdGZvcm1fZGlycyhwb3J0 KSwgWyd0ZXN0d2Via2l0cG9ydCddKQogCi0gICAgICAgIHBvcnQuX25hbWUgPSAidGVzdHdlYmtp dHBvcnQtdmVyc2lvbiIKKyAgICAgICAgcG9ydCA9IFRlc3RXZWJLaXRQb3J0KHBvcnRfbmFtZT0i dGVzdHdlYmtpdHBvcnQtdmVyc2lvbiIpCiAgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwocGxhdGZv cm1fZGlycyhwb3J0KSwgWyd0ZXN0d2Via2l0cG9ydCcsICd0ZXN0d2Via2l0cG9ydC12ZXJzaW9u J10pCiAKLSAgICAgICAgcG9ydC5fb3B0aW9ucyA9IE1vY2tPcHRpb25zKHdlYmtpdF90ZXN0X3J1 bm5lcj1UcnVlKQotICAgICAgICBzZWxmLmFzc2VydEVxdWFsKHBsYXRmb3JtX2RpcnMocG9ydCks IFsndGVzdHdlYmtpdHBvcnQnLCAndGVzdHdlYmtpdHBvcnQtdmVyc2lvbicsICd0ZXN0d2Via2l0 cG9ydC13azInLCAnd2syJ10pCisgICAgICAgIHBvcnQgPSBUZXN0V2ViS2l0UG9ydChwb3J0X25h bWU9InRlc3R3ZWJraXRwb3J0LXZlcnNpb24td2syIikKKyAgICAgICAgc2VsZi5hc3NlcnRFcXVh bChwbGF0Zm9ybV9kaXJzKHBvcnQpLCBbJ3Rlc3R3ZWJraXRwb3J0JywgJ3Rlc3R3ZWJraXRwb3J0 LXZlcnNpb24nLCAnd2syJywgJ3Rlc3R3ZWJraXRwb3J0LXdrMiddKQogCi0gICAgICAgIHBvcnQu X29wdGlvbnMgPSBNb2NrT3B0aW9ucyhhZGRpdGlvbmFsX3BsYXRmb3JtX2RpcmVjdG9yeT1bImlu dGVybmFsLXRlc3R3ZWJraXRwb3J0Il0pCisgICAgICAgIHBvcnQgPSBUZXN0V2ViS2l0UG9ydChw b3J0X25hbWU9InRlc3R3ZWJraXRwb3J0LXZlcnNpb24iLAorICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgb3B0aW9ucz1Nb2NrT3B0aW9ucyhhZGRpdGlvbmFsX3BsYXRmb3JtX2RpcmVjdG9y eT1bImludGVybmFsLXRlc3R3ZWJraXRwb3J0Il0pKQogICAgICAgICBzZWxmLmFzc2VydEVxdWFs KHBsYXRmb3JtX2RpcnMocG9ydCksIFsndGVzdHdlYmtpdHBvcnQnLCAndGVzdHdlYmtpdHBvcnQt dmVyc2lvbicsICdpbnRlcm5hbC10ZXN0d2Via2l0cG9ydCddKQogCiAgICAgZGVmIHRlc3Rfcm9v dF9vcHRpb24oc2VsZik6Cg==