204174 2019-11-13 15:33:08 -0800 [ews] Add build step to Analyze JSC Tests Results 2019-11-13 19:08:19 -0800 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=204179 InRadar P2 Normal --- 1 aakash_jain aakash_jain aakash_jain ap commit-queue jbedard pmatos webkit-bug-importer zhifei_fang oldest_to_newest 1590068 0 aakash_jain 2019-11-13 15:33:08 -0800 Add build step to analyze JavaScriptCore tests results, including re-run and clean-tree-run. Determine if the failures were introduced by the patch, were pre-existing. 1590070 1 383502 aakash_jain 2019-11-13 15:38:59 -0800 Created attachment 383502 Patch 1590080 2 aakash_jain 2019-11-13 15:56:04 -0800 Sample runs: JSC stress test failure: https://ews-build.webkit-uat.org/#/builders/21/builds/430 binary and stress test failure: https://ews-build.webkit-uat.org/#/builders/21/builds/442 Flaky tests: https://ews-build.webkit-uat.org/#/builders/21/builds/435 Pre-existing failure: https://ews-build.webkit-uat.org/#/builders/21/builds/457 1590129 3 383502 jbedard 2019-11-13 17:21:37 -0800 Comment on attachment 383502 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=383502&action=review > Tools/BuildSlaveSupport/ews-build/steps.py:1081 > + flaky_binary_failures = first_run_binary_failures.union(second_run_binary_failures) - first_run_binary_failures.intersection(second_run_binary_failures) I'm not convinced that we should give a green bubble with flaky binary failures....From talking to Zhifei, I get the sense that the binaries don't have the same sort of flake problems that layout tests or jsc stress tests have. 1590130 4 zhifei_fang 2019-11-13 17:24:39 -0800 wait, if we have already submitted results to results database, why we need to rerun it without patch ? 1590136 5 jbedard 2019-11-13 17:35:36 -0800 (In reply to Zhifei Fang from comment #4) > wait, if we have already submitted results to results database, why we need > to rerun it without patch ? We don't have EWS plugged in to the results database yet. We totally could now, though. Ling recently sorted out the network access. 1590138 6 383502 jbedard 2019-11-13 17:37:47 -0800 Comment on attachment 383502 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=383502&action=review >> Tools/BuildSlaveSupport/ews-build/steps.py:1081 >> + flaky_binary_failures = first_run_binary_failures.union(second_run_binary_failures) - first_run_binary_failures.intersection(second_run_binary_failures) > > I'm not convinced that we should give a green bubble with flaky binary failures....From talking to Zhifei, I get the sense that the binaries don't have the same sort of flake problems that layout tests or jsc stress tests have. Aakash, Zhifei and I talked about this a bit offline. Binary tests probably aren't flakey, but this is what old EWS did and we don't actually have concrete data on if test binaries are flakey without https://bugs.webkit.org/show_bug.cgi?id=204091. We might come back and revisit this, but for the sake of converting JSC EWS to new EWS, this behavior is the same as what we were doing before. 1590170 7 383502 commit-queue 2019-11-13 19:07:41 -0800 Comment on attachment 383502 Patch Clearing flags on attachment: 383502 Committed r252446: <https://trac.webkit.org/changeset/252446> 1590171 8 commit-queue 2019-11-13 19:07:43 -0800 All reviewed patches have been landed. Closing bug. 1590173 9 webkit-bug-importer 2019-11-13 19:08:19 -0800 <rdar://problem/57177809> 383502 2019-11-13 15:38:59 -0800 2019-11-13 19:07:41 -0800 Patch bug-204174-20191113183857.patch text/plain 14734 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI1MjQzNSkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDI3IEBACisyMDE5LTExLTEzICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBbZXdzXSBBZGQgYnVpbGQgc3RlcCB0byBBbmFseXplIEpTQyBUZXN0cyBS ZXN1bHRzCisgICAgICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD0y MDQxNzQKKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAgICAq IEJ1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwcy5weToKKyAgICAgICAgKFJlUnVuSmF2 YVNjcmlwdENvcmVUZXN0cy5ldmFsdWF0ZUNvbW1hbmQpOiBpbnZva2UgQW5hbHl6ZUpTQ1Rlc3Rz UmVzdWx0cyBzdGVwLgorICAgICAgICAoQW5hbHl6ZUpTQ1Rlc3RzUmVzdWx0cyk6IEJ1aWxkIHN0 ZXAgdG8gYW5hbHl6ZSBsYXlvdXQtdGVzdCByZXN1bHRzLgorICAgICAgICAqIEJ1aWxkU2xhdmVT dXBwb3J0L2V3cy1idWlsZC9zdGVwc191bml0dGVzdC5weToKKyAgICAgICAgKFRlc3RBbmFseXpl SlNDVGVzdHNSZXN1bHRzKToKKyAgICAgICAgKFRlc3RBbmFseXplSlNDVGVzdHNSZXN1bHRzLnRl c3Rfc2luZ2xlX25ld19zdHJlc3NfZmFpbHVyZSk6IEFkZGVkIHVuaXQtdGVzdC4KKyAgICAgICAg KFRlc3RBbmFseXplSlNDVGVzdHNSZXN1bHRzLnRlc3Rfc2luZ2xlX25ld19iaW5hcnlfZmFpbHVy ZSk6IERpdHRvLgorICAgICAgICAoVGVzdEFuYWx5emVKU0NUZXN0c1Jlc3VsdHMudGVzdF9tdWx0 aXBsZV9uZXdfc3RyZXNzX2ZhaWx1cmUpOiBEaXR0by4KKyAgICAgICAgKFRlc3RBbmFseXplSlND VGVzdHNSZXN1bHRzLnRlc3RfbXVsdGlwbGVfbmV3X2JpbmFyeV9mYWlsdXJlKTogRGl0dG8uCisg ICAgICAgIChUZXN0QW5hbHl6ZUpTQ1Rlc3RzUmVzdWx0cy50ZXN0X25ld19zdHJlc3NfYW5kX2Jp bmFyeV9mYWlsdXJlKTogRGl0dG8uCisgICAgICAgIChUZXN0QW5hbHl6ZUpTQ1Rlc3RzUmVzdWx0 cy50ZXN0X3N0cmVzc19mYWlsdXJlX29uX2NsZWFuX3RyZWUpOiBEaXR0by4KKyAgICAgICAgKFRl c3RBbmFseXplSlNDVGVzdHNSZXN1bHRzLnRlc3RfYmluYXJ5X2ZhaWx1cmVfb25fY2xlYW5fdHJl ZSk6IERpdHRvLgorICAgICAgICAoVGVzdEFuYWx5emVKU0NUZXN0c1Jlc3VsdHMudGVzdF9zdHJl c3NfYW5kX2JpbmFyeV9mYWlsdXJlX29uX2NsZWFuX3RyZWUpOiBEaXR0by4KKyAgICAgICAgKFRl c3RBbmFseXplSlNDVGVzdHNSZXN1bHRzLnRlc3RfZmxha3lfc3RyZXNzX2FuZF9iaW5hcnlfZmFp bHVyZXMpOiBEaXR0by4KKyAgICAgICAgKFRlc3RBbmFseXplSlNDVGVzdHNSZXN1bHRzLnRlc3Rf Zmxha3lfYW5kX2NvbnNpc3RlbnRfc3RyZXNzX2ZhaWx1cmVzKTogRGl0dG8uCisgICAgICAgIChU ZXN0QW5hbHl6ZUpTQ1Rlc3RzUmVzdWx0cy50ZXN0X2ZsYWt5X2FuZF9jb25zaXN0ZW50X2ZhaWx1 cmVzX3dpdGhfY2xlYW5fdHJlZV9mYWlsdXJlcyk6IERpdHRvLgorCiAyMDE5LTExLTEzICBNeWxl cyBDLiBNYXhmaWVsZCAgPG1tYXhmaWVsZEBhcHBsZS5jb20+CiAKICAgICAgICAgW01hY10gRml4 IGJ1aWxkCkluZGV4OiBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9ld3MtYnVpbGQvc3RlcHMucHkK PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PQotLS0gVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvZXdzLWJ1aWxkL3N0ZXBzLnB5 CShyZXZpc2lvbiAyNTI0MzApCisrKyBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9ld3MtYnVpbGQv c3RlcHMucHkJKHdvcmtpbmcgY29weSkKQEAgLTEwNDYsNyArMTA0Niw3IEBAIGNsYXNzIFJlUnVu SmF2YVNjcmlwdENvcmVUZXN0cyhSdW5KYXZhU2MKICAgICAgICAgICAgIHNlbGYuYnVpbGQuYnVp bGRGaW5pc2hlZChbbWVzc2FnZV0sIFNVQ0NFU1MpCiAgICAgICAgIGVsc2U6CiAgICAgICAgICAg ICBzZWxmLnNldFByb3BlcnR5KCdwYXRjaEZhaWxlZFRlc3RzJywgVHJ1ZSkKLSAgICAgICAgICAg IHNlbGYuYnVpbGQuYWRkU3RlcHNBZnRlckN1cnJlbnRTdGVwKFtVbkFwcGx5UGF0Y2hJZlJlcXVp cmVkKCksIENvbXBpbGVKU0NUb1QoKSwgUnVuSlNDVGVzdHNXaXRob3V0UGF0Y2goKV0pCisgICAg ICAgICAgICBzZWxmLmJ1aWxkLmFkZFN0ZXBzQWZ0ZXJDdXJyZW50U3RlcChbVW5BcHBseVBhdGNo SWZSZXF1aXJlZCgpLCBDb21waWxlSlNDVG9UKCksIFJ1bkpTQ1Rlc3RzV2l0aG91dFBhdGNoKCks IEFuYWx5emVKU0NUZXN0c1Jlc3VsdHMoKV0pCiAgICAgICAgIHJldHVybiByYwogCiAKQEAgLTEw NTgsNiArMTA1OCw4MiBAQCBjbGFzcyBSdW5KU0NUZXN0c1dpdGhvdXRQYXRjaChSdW5KYXZhU2Ny CiAgICAgICAgIHJldHVybiBzaGVsbC5UZXN0LmV2YWx1YXRlQ29tbWFuZChzZWxmLCBjbWQpCiAK IAorY2xhc3MgQW5hbHl6ZUpTQ1Rlc3RzUmVzdWx0cyhidWlsZHN0ZXAuQnVpbGRTdGVwKToKKyAg ICBuYW1lID0gJ2FuYWx5emUtanNjLXRlc3RzLXJlc3VsdHMnCisgICAgZGVzY3JpcHRpb24gPSBb J2FuYWx5emUtanNjLXRlc3QtcmVzdWx0cyddCisgICAgZGVzY3JpcHRpb25Eb25lID0gWydhbmFs eXplLWpzYy10ZXN0cy1yZXN1bHRzJ10KKyAgICBOVU1fRkFJTFVSRVNfVE9fRElTUExBWSA9IDEw CisKKyAgICBkZWYgc3RhcnQoc2VsZik6CisgICAgICAgIGZpcnN0X3J1bl9zdHJlc3NfZmFpbHVy ZXMgPSBzZXQoc2VsZi5nZXRQcm9wZXJ0eSgnanNjX3N0cmVzc190ZXN0X2ZhaWx1cmVzJywgW10p KQorICAgICAgICBmaXJzdF9ydW5fYmluYXJ5X2ZhaWx1cmVzID0gc2V0KHNlbGYuZ2V0UHJvcGVy dHkoJ2pzY19iaW5hcnlfZmFpbHVyZXMnLCBbXSkpCisgICAgICAgIHNlY29uZF9ydW5fc3RyZXNz X2ZhaWx1cmVzID0gc2V0KHNlbGYuZ2V0UHJvcGVydHkoJ2pzY19yZXJ1bl9zdHJlc3NfdGVzdF9m YWlsdXJlcycsIFtdKSkKKyAgICAgICAgc2Vjb25kX3J1bl9iaW5hcnlfZmFpbHVyZXMgPSBzZXQo c2VsZi5nZXRQcm9wZXJ0eSgnanNjX3JlcnVuX2JpbmFyeV9mYWlsdXJlcycsIFtdKSkKKyAgICAg ICAgY2xlYW5fdHJlZV9zdHJlc3NfZmFpbHVyZXMgPSBzZXQoc2VsZi5nZXRQcm9wZXJ0eSgnanNj X2NsZWFuX3RyZWVfc3RyZXNzX3Rlc3RfZmFpbHVyZXMnLCBbXSkpCisgICAgICAgIGNsZWFuX3Ry ZWVfYmluYXJ5X2ZhaWx1cmVzID0gc2V0KHNlbGYuZ2V0UHJvcGVydHkoJ2pzY19jbGVhbl90cmVl X2JpbmFyeV9mYWlsdXJlcycsIFtdKSkKKyAgICAgICAgY2xlYW5fdHJlZV9mYWlsdXJlcyA9IGxp c3QoY2xlYW5fdHJlZV9iaW5hcnlfZmFpbHVyZXMpICsgbGlzdChjbGVhbl90cmVlX3N0cmVzc19m YWlsdXJlcykKKyAgICAgICAgY2xlYW5fdHJlZV9mYWlsdXJlc19zdHJpbmcgPSAnLCAnLmpvaW4o Y2xlYW5fdHJlZV9mYWlsdXJlcykKKworICAgICAgICBzdHJlc3NfZmFpbHVyZXNfd2l0aF9wYXRj aCA9IGZpcnN0X3J1bl9zdHJlc3NfZmFpbHVyZXMuaW50ZXJzZWN0aW9uKHNlY29uZF9ydW5fc3Ry ZXNzX2ZhaWx1cmVzKQorICAgICAgICBiaW5hcnlfZmFpbHVyZXNfd2l0aF9wYXRjaCA9IGZpcnN0 X3J1bl9iaW5hcnlfZmFpbHVyZXMuaW50ZXJzZWN0aW9uKHNlY29uZF9ydW5fYmluYXJ5X2ZhaWx1 cmVzKQorCisgICAgICAgIGZsYWt5X3N0cmVzc19mYWlsdXJlcyA9IGZpcnN0X3J1bl9zdHJlc3Nf ZmFpbHVyZXMudW5pb24oc2Vjb25kX3J1bl9zdHJlc3NfZmFpbHVyZXMpIC0gZmlyc3RfcnVuX3N0 cmVzc19mYWlsdXJlcy5pbnRlcnNlY3Rpb24oc2Vjb25kX3J1bl9zdHJlc3NfZmFpbHVyZXMpCisg ICAgICAgIGZsYWt5X2JpbmFyeV9mYWlsdXJlcyA9IGZpcnN0X3J1bl9iaW5hcnlfZmFpbHVyZXMu dW5pb24oc2Vjb25kX3J1bl9iaW5hcnlfZmFpbHVyZXMpIC0gZmlyc3RfcnVuX2JpbmFyeV9mYWls dXJlcy5pbnRlcnNlY3Rpb24oc2Vjb25kX3J1bl9iaW5hcnlfZmFpbHVyZXMpCisgICAgICAgIGZs YWt5X2ZhaWx1cmVzID0gbGlzdChmbGFreV9iaW5hcnlfZmFpbHVyZXMpICsgbGlzdChmbGFreV9z dHJlc3NfZmFpbHVyZXMpCisgICAgICAgIGZsYWt5X2ZhaWx1cmVzX3N0cmluZyA9ICcsICcuam9p bihmbGFreV9mYWlsdXJlcykKKworICAgICAgICBuZXdfc3RyZXNzX2ZhaWx1cmVzID0gc3RyZXNz X2ZhaWx1cmVzX3dpdGhfcGF0Y2ggLSBjbGVhbl90cmVlX3N0cmVzc19mYWlsdXJlcworICAgICAg ICBuZXdfYmluYXJ5X2ZhaWx1cmVzID0gYmluYXJ5X2ZhaWx1cmVzX3dpdGhfcGF0Y2ggLSBjbGVh bl90cmVlX2JpbmFyeV9mYWlsdXJlcworICAgICAgICBuZXdfc3RyZXNzX2ZhaWx1cmVzX3RvX2Rp c3BsYXkgPSAnLCAnLmpvaW4obGlzdChuZXdfc3RyZXNzX2ZhaWx1cmVzKVs6c2VsZi5OVU1fRkFJ TFVSRVNfVE9fRElTUExBWV0pCisgICAgICAgIG5ld19iaW5hcnlfZmFpbHVyZXNfdG9fZGlzcGxh eSA9ICcsICcuam9pbihsaXN0KG5ld19iaW5hcnlfZmFpbHVyZXMpWzpzZWxmLk5VTV9GQUlMVVJF U19UT19ESVNQTEFZXSkKKworICAgICAgICBzZWxmLl9hZGRUb0xvZygnc3RkZXJyJywgJ1xuRmFp bHVyZXMgaW4gZmlyc3QgcnVuOiB7fScuZm9ybWF0KGxpc3QoZmlyc3RfcnVuX2JpbmFyeV9mYWls dXJlcykgKyBsaXN0KGZpcnN0X3J1bl9zdHJlc3NfZmFpbHVyZXMpKSkKKyAgICAgICAgc2VsZi5f YWRkVG9Mb2coJ3N0ZGVycicsICdcbkZhaWx1cmVzIGluIHNlY29uZCBydW46IHt9Jy5mb3JtYXQo bGlzdChzZWNvbmRfcnVuX2JpbmFyeV9mYWlsdXJlcykgKyBsaXN0KHNlY29uZF9ydW5fc3RyZXNz X2ZhaWx1cmVzKSkpCisgICAgICAgIHNlbGYuX2FkZFRvTG9nKCdzdGRlcnInLCAnXG5GbGFreSBU ZXN0czoge30nLmZvcm1hdChmbGFreV9mYWlsdXJlc19zdHJpbmcpKQorICAgICAgICBzZWxmLl9h ZGRUb0xvZygnc3RkZXJyJywgJ1xuRmFpbHVyZXMgb24gY2xlYW4gdHJlZToge30nLmZvcm1hdChs aXN0KGNsZWFuX3RyZWVfc3RyZXNzX2ZhaWx1cmVzKSArIGxpc3QoY2xlYW5fdHJlZV9iaW5hcnlf ZmFpbHVyZXMpKSkKKworICAgICAgICBpZiBuZXdfc3RyZXNzX2ZhaWx1cmVzIG9yIG5ld19iaW5h cnlfZmFpbHVyZXM6CisgICAgICAgICAgICBzZWxmLl9hZGRUb0xvZygnc3RkZXJyJywgJ1xuTmV3 IGZhaWx1cmVzOiB7fVxuJy5mb3JtYXQobGlzdChuZXdfYmluYXJ5X2ZhaWx1cmVzKSArIGxpc3Qo bmV3X3N0cmVzc19mYWlsdXJlcykpKQorICAgICAgICAgICAgc2VsZi5maW5pc2hlZChGQUlMVVJF KQorICAgICAgICAgICAgc2VsZi5idWlsZC5yZXN1bHRzID0gRkFJTFVSRQorICAgICAgICAgICAg bWVzc2FnZSA9ICcnCisgICAgICAgICAgICBpZiBuZXdfYmluYXJ5X2ZhaWx1cmVzOgorICAgICAg ICAgICAgICAgIHBsdXJhbFN1ZmZpeCA9ICdzJyBpZiBsZW4obmV3X2JpbmFyeV9mYWlsdXJlcykg PiAxIGVsc2UgJycKKyAgICAgICAgICAgICAgICBtZXNzYWdlID0gJ0ZvdW5kIHt9IG5ldyBKU0Mg YmluYXJ5IGZhaWx1cmV7fToge30nLmZvcm1hdChsZW4obmV3X2JpbmFyeV9mYWlsdXJlcyksIHBs dXJhbFN1ZmZpeCwgbmV3X2JpbmFyeV9mYWlsdXJlc190b19kaXNwbGF5KQorICAgICAgICAgICAg aWYgbmV3X3N0cmVzc19mYWlsdXJlczoKKyAgICAgICAgICAgICAgICBpZiBtZXNzYWdlOgorICAg ICAgICAgICAgICAgICAgICBtZXNzYWdlICs9ICcsICcKKyAgICAgICAgICAgICAgICBwbHVyYWxT dWZmaXggPSAncycgaWYgbGVuKG5ld19zdHJlc3NfZmFpbHVyZXMpID4gMSBlbHNlICcnCisgICAg ICAgICAgICAgICAgbWVzc2FnZSArPSAnRm91bmQge30gbmV3IEpTQyBzdHJlc3MgdGVzdCBmYWls dXJle306IHt9Jy5mb3JtYXQobGVuKG5ld19zdHJlc3NfZmFpbHVyZXMpLCBwbHVyYWxTdWZmaXgs IG5ld19zdHJlc3NfZmFpbHVyZXNfdG9fZGlzcGxheSkKKyAgICAgICAgICAgICAgICBpZiBsZW4o bmV3X3N0cmVzc19mYWlsdXJlcykgPiBzZWxmLk5VTV9GQUlMVVJFU19UT19ESVNQTEFZOgorICAg ICAgICAgICAgICAgICAgICBtZXNzYWdlICs9ICcgLi4uJworICAgICAgICAgICAgc2VsZi5kZXNj cmlwdGlvbkRvbmUgPSBtZXNzYWdlCisgICAgICAgICAgICBzZWxmLmJ1aWxkLmJ1aWxkRmluaXNo ZWQoW21lc3NhZ2VdLCBGQUlMVVJFKQorICAgICAgICBlbHNlOgorICAgICAgICAgICAgc2VsZi5f YWRkVG9Mb2coJ3N0ZGVycicsICdcbk5vIG5ldyBmYWlsdXJlc1xuJykKKyAgICAgICAgICAgIHNl bGYuZmluaXNoZWQoU1VDQ0VTUykKKyAgICAgICAgICAgIHNlbGYuYnVpbGQucmVzdWx0cyA9IFNV Q0NFU1MKKyAgICAgICAgICAgIHNlbGYuZGVzY3JpcHRpb25Eb25lID0gJ1Bhc3NlZCBKU0MgdGVz dHMnCisgICAgICAgICAgICBwbHVyYWxTdWZmaXggPSAncycgaWYgbGVuKGNsZWFuX3RyZWVfZmFp bHVyZXMpID4gMSBlbHNlICcnCisgICAgICAgICAgICBtZXNzYWdlID0gJycKKyAgICAgICAgICAg IGlmIGNsZWFuX3RyZWVfZmFpbHVyZXM6CisgICAgICAgICAgICAgICAgbWVzc2FnZSA9ICdGb3Vu ZCB7fSBwcmUtZXhpc3RpbmcgSlNDIHRlc3QgZmFpbHVyZXt9OiB7fScuZm9ybWF0KGxlbihjbGVh bl90cmVlX2ZhaWx1cmVzKSwgcGx1cmFsU3VmZml4LCBjbGVhbl90cmVlX2ZhaWx1cmVzX3N0cmlu ZykKKyAgICAgICAgICAgIGlmIGxlbihjbGVhbl90cmVlX2ZhaWx1cmVzKSA+IHNlbGYuTlVNX0ZB SUxVUkVTX1RPX0RJU1BMQVk6CisgICAgICAgICAgICAgICAgbWVzc2FnZSArPSAnIC4uLicKKyAg ICAgICAgICAgIGlmIGZsYWt5X2ZhaWx1cmVzOgorICAgICAgICAgICAgICAgIG1lc3NhZ2UgKz0g JyBGb3VuZCBmbGFreSB0ZXN0czoge30nLmZvcm1hdChmbGFreV9mYWlsdXJlc19zdHJpbmcpCisg ICAgICAgICAgICBzZWxmLmJ1aWxkLmJ1aWxkRmluaXNoZWQoW21lc3NhZ2VdLCBTVUNDRVNTKQor ICAgICAgICByZXR1cm4gZGVmZXIuc3VjY2VlZChOb25lKQorCisgICAgQGRlZmVyLmlubGluZUNh bGxiYWNrcworICAgIGRlZiBfYWRkVG9Mb2coc2VsZiwgbG9nTmFtZSwgbWVzc2FnZSk6CisgICAg ICAgIHRyeToKKyAgICAgICAgICAgIGxvZyA9IHNlbGYuZ2V0TG9nKGxvZ05hbWUpCisgICAgICAg IGV4Y2VwdCBLZXlFcnJvcjoKKyAgICAgICAgICAgIGxvZyA9IHlpZWxkIHNlbGYuYWRkTG9nKGxv Z05hbWUpCisgICAgICAgIGxvZy5hZGRTdGRvdXQobWVzc2FnZSkKKworCiBjbGFzcyBDbGVhbkJ1 aWxkKHNoZWxsLkNvbXBpbGUpOgogICAgIG5hbWUgPSAnZGVsZXRlLVdlYktpdEJ1aWxkLWRpcmVj dG9yeScKICAgICBkZXNjcmlwdGlvbiA9IFsnZGVsZXRpbmcgV2ViS2l0QnVpbGQgZGlyZWN0b3J5 J10KSW5kZXg6IFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwc191bml0dGVz dC5weQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09Ci0tLSBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9ld3MtYnVpbGQvc3Rl cHNfdW5pdHRlc3QucHkJKHJldmlzaW9uIDI1MjQzMCkKKysrIFRvb2xzL0J1aWxkU2xhdmVTdXBw b3J0L2V3cy1idWlsZC9zdGVwc191bml0dGVzdC5weQkod29ya2luZyBjb3B5KQpAQCAtMzQsNyAr MzQsNyBAQCBmcm9tIHR3aXN0ZWQuaW50ZXJuZXQgaW1wb3J0IGVycm9yLCByZWFjCiBmcm9tIHR3 aXN0ZWQucHl0aG9uIGltcG9ydCBmYWlsdXJlLCBsb2cKIGZyb20gdHdpc3RlZC50cmlhbCBpbXBv cnQgdW5pdHRlc3QKIAotZnJvbSBzdGVwcyBpbXBvcnQgKEFuYWx5emVBUElUZXN0c1Jlc3VsdHMs IEFuYWx5emVDb21waWxlV2ViS2l0UmVzdWx0cywgQW5hbHl6ZUxheW91dFRlc3RzUmVzdWx0cywg QXBwbHlQYXRjaCwgQXBwbHlXYXRjaExpc3QsIEFyY2hpdmVCdWlsdFByb2R1Y3QsIEFyY2hpdmVU ZXN0UmVzdWx0cywKK2Zyb20gc3RlcHMgaW1wb3J0IChBbmFseXplQVBJVGVzdHNSZXN1bHRzLCBB bmFseXplQ29tcGlsZVdlYktpdFJlc3VsdHMsIEFuYWx5emVKU0NUZXN0c1Jlc3VsdHMsIEFuYWx5 emVMYXlvdXRUZXN0c1Jlc3VsdHMsIEFwcGx5UGF0Y2gsIEFwcGx5V2F0Y2hMaXN0LCBBcmNoaXZl QnVpbHRQcm9kdWN0LCBBcmNoaXZlVGVzdFJlc3VsdHMsCiAgICAgICAgICAgICAgICAgICAgQ2hl Y2tPdXRTb3VyY2UsIENoZWNrT3V0U3BlY2lmaWNSZXZpc2lvbiwgQ2hlY2tQYXRjaFJlbGV2YW5j ZSwgQ2hlY2tTdHlsZSwgQ2xlYW5CdWlsZCwgQ2xlYW5VcEdpdEluZGV4TG9jaywgQ2xlYW5Xb3Jr aW5nRGlyZWN0b3J5LAogICAgICAgICAgICAgICAgICAgIENvbXBpbGVKU0MsIENvbXBpbGVKU0NU b1QsIENvbXBpbGVXZWJLaXQsIENvbXBpbGVXZWJLaXRUb1QsIENvbmZpZ3VyZUJ1aWxkLAogICAg ICAgICAgICAgICAgICAgIERvd25sb2FkQnVpbHRQcm9kdWN0LCBEb3dubG9hZEJ1aWx0UHJvZHVj dEZyb21NYXN0ZXIsIEV4dHJhY3RCdWlsdFByb2R1Y3QsIEV4dHJhY3RUZXN0UmVzdWx0cywgSW5z dGFsbEd0a0RlcGVuZGVuY2llcywgSW5zdGFsbFdwZURlcGVuZGVuY2llcywgS2lsbE9sZFByb2Nl c3NlcywKQEAgLTExOTksNiArMTE5OSwxMTAgQEAgY2xhc3MgVGVzdFJ1bkpTQ1Rlc3RzV2l0aG91 dFBhdGNoKEJ1aWxkUwogICAgICAgICByZXR1cm4gc2VsZi5ydW5TdGVwKCkKIAogCitjbGFzcyBU ZXN0QW5hbHl6ZUpTQ1Rlc3RzUmVzdWx0cyhCdWlsZFN0ZXBNaXhpbkFkZGl0aW9ucywgdW5pdHRl c3QuVGVzdENhc2UpOgorICAgIGRlZiBzZXRVcChzZWxmKToKKyAgICAgICAgc2VsZi5sb25nTWVz c2FnZSA9IFRydWUKKyAgICAgICAgcmV0dXJuIHNlbGYuc2V0VXBCdWlsZFN0ZXAoKQorCisgICAg ZGVmIHRlYXJEb3duKHNlbGYpOgorICAgICAgICByZXR1cm4gc2VsZi50ZWFyRG93bkJ1aWxkU3Rl cCgpCisKKyAgICBkZWYgY29uZmlndXJlU3RlcChzZWxmKToKKyAgICAgICAgc2VsZi5zZXR1cFN0 ZXAoQW5hbHl6ZUpTQ1Rlc3RzUmVzdWx0cygpKQorICAgICAgICBzZWxmLnNldFByb3BlcnR5KCdq c2Nfc3RyZXNzX3Rlc3RfZmFpbHVyZXMnLCBbXSkKKyAgICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgn anNjX2JpbmFyeV9mYWlsdXJlcycsIFtdKQorICAgICAgICBzZWxmLnNldFByb3BlcnR5KCdqc2Nf cmVydW5fc3RyZXNzX3Rlc3RfZmFpbHVyZXMnLCBbXSkKKyAgICAgICAgc2VsZi5zZXRQcm9wZXJ0 eSgnanNjX3JlcnVuX2JpbmFyeV9mYWlsdXJlcycsIFtdKQorICAgICAgICBzZWxmLnNldFByb3Bl cnR5KCdqc2NfY2xlYW5fdHJlZV9zdHJlc3NfdGVzdF9mYWlsdXJlcycsIFtdKQorICAgICAgICBz ZWxmLnNldFByb3BlcnR5KCdqc2NfY2xlYW5fdHJlZV9iaW5hcnlfZmFpbHVyZXMnLCBbXSkKKwor ICAgIGRlZiB0ZXN0X3NpbmdsZV9uZXdfc3RyZXNzX2ZhaWx1cmUoc2VsZik6CisgICAgICAgIHNl bGYuY29uZmlndXJlU3RlcCgpCisgICAgICAgIHNlbGYuc2V0UHJvcGVydHkoJ2pzY19zdHJlc3Nf dGVzdF9mYWlsdXJlcycsIFsnc3RyZXNzL2ZvcmNlLWVycm9yLmpzLmJ5dGVjb2RlLWNhY2hlJ10p CisgICAgICAgIHNlbGYuc2V0UHJvcGVydHkoJ2pzY19yZXJ1bl9zdHJlc3NfdGVzdF9mYWlsdXJl cycsIFsnc3RyZXNzL2ZvcmNlLWVycm9yLmpzLmJ5dGVjb2RlLWNhY2hlJ10pCisgICAgICAgIHNl bGYuZXhwZWN0T3V0Y29tZShyZXN1bHQ9RkFJTFVSRSwgc3RhdGVfc3RyaW5nPSdGb3VuZCAxIG5l dyBKU0Mgc3RyZXNzIHRlc3QgZmFpbHVyZTogc3RyZXNzL2ZvcmNlLWVycm9yLmpzLmJ5dGVjb2Rl LWNhY2hlIChmYWlsdXJlKScpCisgICAgICAgIHJldHVybiBzZWxmLnJ1blN0ZXAoKQorCisgICAg ZGVmIHRlc3Rfc2luZ2xlX25ld19iaW5hcnlfZmFpbHVyZShzZWxmKToKKyAgICAgICAgc2VsZi5j b25maWd1cmVTdGVwKCkKKyAgICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgnanNjX2JpbmFyeV9mYWls dXJlcycsIFsndGVzdG1hc20nXSkKKyAgICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgnanNjX3JlcnVu X2JpbmFyeV9mYWlsdXJlcycsIFsndGVzdG1hc20nXSkKKyAgICAgICAgc2VsZi5leHBlY3RPdXRj b21lKHJlc3VsdD1GQUlMVVJFLCBzdGF0ZV9zdHJpbmc9J0ZvdW5kIDEgbmV3IEpTQyBiaW5hcnkg ZmFpbHVyZTogdGVzdG1hc20gKGZhaWx1cmUpJykKKyAgICAgICAgcmV0dXJuIHNlbGYucnVuU3Rl cCgpCisKKyAgICBkZWYgdGVzdF9tdWx0aXBsZV9uZXdfc3RyZXNzX2ZhaWx1cmUoc2VsZik6Cisg ICAgICAgIHNlbGYuY29uZmlndXJlU3RlcCgpCisgICAgICAgIHNlbGYuc2V0UHJvcGVydHkoJ2pz Y19zdHJlc3NfdGVzdF9mYWlsdXJlcycsIFsndGVzdHt9Jy5mb3JtYXQoaSkgZm9yIGkgaW4gcmFu Z2UoMCwgMzApXSkKKyAgICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgnanNjX3JlcnVuX3N0cmVzc190 ZXN0X2ZhaWx1cmVzJywgWyd0ZXN0e30nLmZvcm1hdChpKSBmb3IgaSBpbiByYW5nZSgwLCAzMCld KQorICAgICAgICBzZWxmLmV4cGVjdE91dGNvbWUocmVzdWx0PUZBSUxVUkUsIHN0YXRlX3N0cmlu Zz0nRm91bmQgMzAgbmV3IEpTQyBzdHJlc3MgdGVzdCBmYWlsdXJlczogdGVzdDEsIHRlc3QwLCB0 ZXN0MywgdGVzdDIsIHRlc3Q1LCB0ZXN0NCwgdGVzdDcsIHRlc3Q2LCB0ZXN0OSwgdGVzdDggLi4u IChmYWlsdXJlKScpCisgICAgICAgIHJldHVybiBzZWxmLnJ1blN0ZXAoKQorCisgICAgZGVmIHRl c3RfbXVsdGlwbGVfbmV3X2JpbmFyeV9mYWlsdXJlKHNlbGYpOgorICAgICAgICBzZWxmLmNvbmZp Z3VyZVN0ZXAoKQorICAgICAgICBzZWxmLnNldFByb3BlcnR5KCdqc2NfYmluYXJ5X2ZhaWx1cmVz JywgWyd0ZXN0bWFzbScsICd0ZXN0YWlyJywgJ3Rlc3RiMycsICd0ZXN0ZGZnJywgJ3Rlc3RhcGkn XSkKKyAgICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgnanNjX3JlcnVuX2JpbmFyeV9mYWlsdXJlcycs IFsndGVzdG1hc20nLCAndGVzdGFpcicsICd0ZXN0YjMnLCAndGVzdGRmZycsICd0ZXN0YXBpJ10p CisgICAgICAgIHNlbGYuZXhwZWN0T3V0Y29tZShyZXN1bHQ9RkFJTFVSRSwgc3RhdGVfc3RyaW5n PSdGb3VuZCA1IG5ldyBKU0MgYmluYXJ5IGZhaWx1cmVzOiB0ZXN0YjMsIHRlc3RtYXNtLCB0ZXN0 YXBpLCB0ZXN0ZGZnLCB0ZXN0YWlyIChmYWlsdXJlKScpCisgICAgICAgIHJldHVybiBzZWxmLnJ1 blN0ZXAoKQorCisgICAgZGVmIHRlc3RfbmV3X3N0cmVzc19hbmRfYmluYXJ5X2ZhaWx1cmUoc2Vs Zik6CisgICAgICAgIHNlbGYuY29uZmlndXJlU3RlcCgpCisgICAgICAgIHNlbGYuc2V0UHJvcGVy dHkoJ2pzY19zdHJlc3NfdGVzdF9mYWlsdXJlcycsIFsnZXM2LnlhbWwvZXM2L1NldF9pdGVyYXRv cl9jbG9zaW5nLmpzLmRlZmF1bHQnXSkKKyAgICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgnanNjX2Jp bmFyeV9mYWlsdXJlcycsIFsndGVzdG1hc20nXSkKKyAgICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgn anNjX3JlcnVuX3N0cmVzc190ZXN0X2ZhaWx1cmVzJywgWydlczYueWFtbC9lczYvU2V0X2l0ZXJh dG9yX2Nsb3NpbmcuanMuZGVmYXVsdCddKQorICAgICAgICBzZWxmLnNldFByb3BlcnR5KCdqc2Nf cmVydW5fYmluYXJ5X2ZhaWx1cmVzJywgWyd0ZXN0bWFzbSddKQorICAgICAgICBzZWxmLmV4cGVj dE91dGNvbWUocmVzdWx0PUZBSUxVUkUsIHN0YXRlX3N0cmluZz0nRm91bmQgMSBuZXcgSlNDIGJp bmFyeSBmYWlsdXJlOiB0ZXN0bWFzbSwgRm91bmQgMSBuZXcgSlNDIHN0cmVzcyB0ZXN0IGZhaWx1 cmU6IGVzNi55YW1sL2VzNi9TZXRfaXRlcmF0b3JfY2xvc2luZy5qcy5kZWZhdWx0IChmYWlsdXJl KScpCisgICAgICAgIHJldHVybiBzZWxmLnJ1blN0ZXAoKQorCisgICAgZGVmIHRlc3Rfc3RyZXNz X2ZhaWx1cmVfb25fY2xlYW5fdHJlZShzZWxmKToKKyAgICAgICAgc2VsZi5jb25maWd1cmVTdGVw KCkKKyAgICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgnanNjX3N0cmVzc190ZXN0X2ZhaWx1cmVzJywg WydzdHJlc3MvZm9yY2UtZXJyb3IuanMuZGVmYXVsdCddKQorICAgICAgICBzZWxmLnNldFByb3Bl cnR5KCdqc2NfcmVydW5fc3RyZXNzX3Rlc3RfZmFpbHVyZXMnLCBbJ3N0cmVzcy9mb3JjZS1lcnJv ci5qcy5kZWZhdWx0J10pCisgICAgICAgIHNlbGYuc2V0UHJvcGVydHkoJ2pzY19jbGVhbl90cmVl X3N0cmVzc190ZXN0X2ZhaWx1cmVzJywgWydzdHJlc3MvZm9yY2UtZXJyb3IuanMuZGVmYXVsdCdd KQorICAgICAgICBzZWxmLmV4cGVjdE91dGNvbWUocmVzdWx0PVNVQ0NFU1MsIHN0YXRlX3N0cmlu Zz0nUGFzc2VkIEpTQyB0ZXN0cycpCisgICAgICAgIHJldHVybiBzZWxmLnJ1blN0ZXAoKQorCisg ICAgZGVmIHRlc3RfYmluYXJ5X2ZhaWx1cmVfb25fY2xlYW5fdHJlZShzZWxmKToKKyAgICAgICAg c2VsZi5jb25maWd1cmVTdGVwKCkKKyAgICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgnanNjX2JpbmFy eV9mYWlsdXJlcycsIFsndGVzdGRmZyddKQorICAgICAgICBzZWxmLnNldFByb3BlcnR5KCdqc2Nf cmVydW5fYmluYXJ5X2ZhaWx1cmVzJywgWyd0ZXN0ZGZnJ10pCisgICAgICAgIHNlbGYuc2V0UHJv cGVydHkoJ2pzY19jbGVhbl90cmVlX2JpbmFyeV9mYWlsdXJlcycsIFsndGVzdGRmZyddKQorICAg ICAgICBzZWxmLmV4cGVjdE91dGNvbWUocmVzdWx0PVNVQ0NFU1MsIHN0YXRlX3N0cmluZz0nUGFz c2VkIEpTQyB0ZXN0cycpCisgICAgICAgIHJldHVybiBzZWxmLnJ1blN0ZXAoKQorCisgICAgZGVm IHRlc3Rfc3RyZXNzX2FuZF9iaW5hcnlfZmFpbHVyZV9vbl9jbGVhbl90cmVlKHNlbGYpOgorICAg ICAgICBzZWxmLmNvbmZpZ3VyZVN0ZXAoKQorICAgICAgICBzZWxmLnNldFByb3BlcnR5KCdqc2Nf c3RyZXNzX3Rlc3RfZmFpbHVyZXMnLCBbJ2VzNi55YW1sL2VzNi9TZXRfaXRlcmF0b3JfY2xvc2lu Zy5qcy5kZWZhdWx0J10pCisgICAgICAgIHNlbGYuc2V0UHJvcGVydHkoJ2pzY19iaW5hcnlfZmFp bHVyZXMnLCBbJ3Rlc3RhaXInXSkKKyAgICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgnanNjX3JlcnVu X3N0cmVzc190ZXN0X2ZhaWx1cmVzJywgWydlczYueWFtbC9lczYvU2V0X2l0ZXJhdG9yX2Nsb3Np bmcuanMuZGVmYXVsdCddKQorICAgICAgICBzZWxmLnNldFByb3BlcnR5KCdqc2NfcmVydW5fYmlu YXJ5X2ZhaWx1cmVzJywgWyd0ZXN0YWlyJ10pCisgICAgICAgIHNlbGYuc2V0UHJvcGVydHkoJ2pz Y19jbGVhbl90cmVlX3N0cmVzc190ZXN0X2ZhaWx1cmVzJywgWydlczYueWFtbC9lczYvU2V0X2l0 ZXJhdG9yX2Nsb3NpbmcuanMuZGVmYXVsdCddKQorICAgICAgICBzZWxmLnNldFByb3BlcnR5KCdq c2NfY2xlYW5fdHJlZV9iaW5hcnlfZmFpbHVyZXMnLCBbJ3Rlc3RhaXInXSkKKyAgICAgICAgc2Vs Zi5leHBlY3RPdXRjb21lKHJlc3VsdD1TVUNDRVNTLCBzdGF0ZV9zdHJpbmc9J1Bhc3NlZCBKU0Mg dGVzdHMnKQorICAgICAgICByZXR1cm4gc2VsZi5ydW5TdGVwKCkKKworICAgIGRlZiB0ZXN0X2Zs YWt5X3N0cmVzc19hbmRfYmluYXJ5X2ZhaWx1cmVzKHNlbGYpOgorICAgICAgICBzZWxmLmNvbmZp Z3VyZVN0ZXAoKQorICAgICAgICBzZWxmLnNldFByb3BlcnR5KCdqc2Nfc3RyZXNzX3Rlc3RfZmFp bHVyZXMnLCBbJ3N0cmVzcy9mb3JjZS1lcnJvci5qcy5kZWZhdWx0J10pCisgICAgICAgIHNlbGYu c2V0UHJvcGVydHkoJ2pzY19iaW5hcnlfZmFpbHVyZXMnLCBbJ3Rlc3RhcGknXSkKKyAgICAgICAg c2VsZi5leHBlY3RPdXRjb21lKHJlc3VsdD1TVUNDRVNTLCBzdGF0ZV9zdHJpbmc9J1Bhc3NlZCBK U0MgdGVzdHMnKQorICAgICAgICByZXR1cm4gc2VsZi5ydW5TdGVwKCkKKworICAgIGRlZiB0ZXN0 X2ZsYWt5X2FuZF9jb25zaXN0ZW50X3N0cmVzc19mYWlsdXJlcyhzZWxmKToKKyAgICAgICAgc2Vs Zi5jb25maWd1cmVTdGVwKCkKKyAgICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgnanNjX3N0cmVzc190 ZXN0X2ZhaWx1cmVzJywgWyd0ZXN0MScsICd0ZXN0MiddKQorICAgICAgICBzZWxmLnNldFByb3Bl cnR5KCdqc2NfcmVydW5fc3RyZXNzX3Rlc3RfZmFpbHVyZXMnLCBbJ3Rlc3QyJ10pCisgICAgICAg IHNlbGYuZXhwZWN0T3V0Y29tZShyZXN1bHQ9RkFJTFVSRSwgc3RhdGVfc3RyaW5nPSdGb3VuZCAx IG5ldyBKU0Mgc3RyZXNzIHRlc3QgZmFpbHVyZTogdGVzdDIgKGZhaWx1cmUpJykKKyAgICAgICAg cmV0dXJuIHNlbGYucnVuU3RlcCgpCisKKyAgICBkZWYgdGVzdF9mbGFreV9hbmRfY29uc2lzdGVu dF9mYWlsdXJlc193aXRoX2NsZWFuX3RyZWVfZmFpbHVyZXMoc2VsZik6CisgICAgICAgIHNlbGYu Y29uZmlndXJlU3RlcCgpCisgICAgICAgIHNlbGYuc2V0UHJvcGVydHkoJ2pzY19zdHJlc3NfdGVz dF9mYWlsdXJlcycsIFsndGVzdDEnLCAndGVzdDInXSkKKyAgICAgICAgc2VsZi5zZXRQcm9wZXJ0 eSgnanNjX3JlcnVuX3N0cmVzc190ZXN0X2ZhaWx1cmVzJywgWyd0ZXN0MSddKQorICAgICAgICBz ZWxmLnNldFByb3BlcnR5KCdqc2NfY2xlYW5fdHJlZV9zdHJlc3NfdGVzdF9mYWlsdXJlcycsIFsn dGVzdDEnLCAndGVzdDInXSkKKyAgICAgICAgc2VsZi5leHBlY3RPdXRjb21lKHJlc3VsdD1TVUND RVNTLCBzdGF0ZV9zdHJpbmc9J1Bhc3NlZCBKU0MgdGVzdHMnKQorICAgICAgICByZXR1cm4gc2Vs Zi5ydW5TdGVwKCkKKworCiBjbGFzcyBUZXN0UnVuV2ViS2l0VGVzdHMoQnVpbGRTdGVwTWl4aW5B ZGRpdGlvbnMsIHVuaXR0ZXN0LlRlc3RDYXNlKToKICAgICBkZWYgc2V0VXAoc2VsZik6CiAgICAg ICAgIHNlbGYubG9uZ01lc3NhZ2UgPSBUcnVlCg==