104858 2012-12-12 16:09:22 -0800 "Running 1 DumpRenderTree over X shards" is not a helpful output 2012-12-14 14:15:01 -0800 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified RESOLVED FIXED P2 Normal --- 1 rniwa rniwa abarth dpranke leviw mjs simon.fraser webkit.review.bot oldest_to_newest 790018 0 rniwa 2012-12-12 16:09:22 -0800 Apparently, "shards" in this context means a list of tests, usually directories. I don't we should be using made up words like "shards" that most of people don't know the exact meaning of. 790039 1 dpranke 2012-12-12 16:20:51 -0800 As discussed, "shard" is not a made up term. As multiple people have told you, it is a common term used when scheduling a set of tasks to be run across multiple physical resources. 790066 2 mjs 2012-12-12 16:46:58 -0800 Alternate name suggestions (for the UI level at least): - chunk - part - section - group - group - subgroup - subset Or any of these (or the word "shard") could be prefixed with "test". I'm sure there's many other possibilities. I think shard makes non-server-experts think of separate (possibly physically separate) computing resources, rather than just pieces of data. Thus, for people without good knowledge of server infrastructure, "over 240 shards" would sound like it's running on 240 different physical/logical servers or 240 different databases, not just running 240 semi-arbitrary chunks of the full test suite. 790072 3 rniwa 2012-12-12 16:56:05 -0800 Also, a typical WebKit contributor doesn’t have an experience working on distributed computing systems. e.g. I graduated from UC Berkeley very recently but I had never heard of the word "shard" in any of my classes except in the context of distributed computing until I joined Google where everyone refers to all sorts of things by "sharding". One thing that makes contributing to WebKit really painful is mounds of technical jargons used in our community like "layout tests", "DRT", "NRWT", etc… We should strive to get rid of them as much as possible so that a reasonably intelligent person can understand various things without having to go through many wiki pages. 790092 4 dpranke 2012-12-12 17:20:41 -0800 jargon is often, but not always, bad. Sometimes inventing new terms to accurately and concisely describe concepts is the right thing to do (I am not necessarily making that claim here, mind you). All of Maciej's suggestions in comment #2 have the downside of being vague and interpreted as one of many possible concepts. In particular, NRWT already uses "part" and "chunk" to refer to different subsets of the test as well. "shard" has the virtue of having fairly specific connotations, even if they are being applied slightly differently here than one might be used to seeing them. As I mentioned over #irc, in the server clustering / cloud community, a shard is *not* necessarily a separate physical resource; shards are virtual concepts that get mapped onto physical resources, typically in an M:N manner (which may of course be 1:1, but the whole point of a shard as opposed to a parallel database is that you can move shards between physical resources to redistribute the load, which is exactly what we're doing here). "batch" or "job" might be better terms that are at least closer in spirit to "shard" than something generic like "section". Unfortunately, we also use --batch-size in NRWT to mean something else. "suite" is also close, but not quiet right (since people might get confused when a suite doesn't refer exactly to a directory). 790096 5 rniwa 2012-12-12 17:22:35 -0800 I’d vote for "test groups". 790394 6 mjs 2012-12-13 01:13:57 -0800 (In reply to comment #4) > jargon is often, but not always, bad. Sometimes inventing new terms to accurately and concisely describe concepts is the right thing to do (I am not necessarily making that claim here, mind you). > > All of Maciej's suggestions in comment #2 have the downside of being vague and interpreted as one of many possible concepts. In particular, NRWT already uses "part" and "chunk" to refer to different subsets of the test as well. "shard" has the virtue of having fairly specific connotations, even if they are being applied slightly differently here than one might be used to seeing them. > > As I mentioned over #irc, in the server clustering / cloud community, a shard is *not* necessarily a separate physical resource; shards are virtual concepts that get mapped onto physical resources, typically in an M:N manner (which may of course be 1:1, but the whole point of a shard as opposed to a parallel database is that you can move shards between physical resources to redistribute the load, which is exactly what we're doing here). I agree that sometimes jargon can be good for its precision, but you have to consider the possibility that a particular term may confuse more than it enlightens. At least four WebKit contributors said on IRC that they actually found this confusing and misinterpreted the message. In light of this, it seems like a poor defense to say that it's technically correct jargon in the server clustering / cloud community. Probably most WebKit contributors are not part of the server clustering / cloud community, nor is there any reason to even assume server clustering is the context here.. (Also, it's kind of crazy that NRWT has so many distinct ways to group the tests that it's hard to find a free term! I am curious what "part" and "chunk" refer to and why they different from "shard".) 790641 7 simon.fraser 2012-12-13 08:42:43 -0800 If a shard to DRT is almost always a directory, why not say: "running DRT in parallel over X directories" or something similar. 790707 8 dpranke 2012-12-13 09:59:12 -0800 (In reply to comment #6) > I agree that sometimes jargon can be good for its precision, but you have to consider the possibility that a particular term may confuse more than it enlightens. Of course. That's why I said I wasn't necessarily making that argument. > At least four WebKit contributors said on IRC that they actually found this confusing and misinterpreted the message. > At least as many people on IRC said that they thought this was a correct and accurate term. And it may be that now that the others have had this explained, they are enlightened. I'm pretty sure that's how learning new vocabulary works. > (Also, it's kind of crazy that NRWT has so many distinct ways to group the tests that it's hard to find a free term! I am curious what "part" and "chunk" refer to and why they different from "shard".) From new-run-webkit-tests --help: --run-chunk=RUN_CHUNK Run a specified chunk (n:l), the nth of len l, of the layout tests --run-part=RUN_PART Run a specified part (n:m), the nth of m parts, of the layout tests --batch-size=BATCH_SIZE Run a the tests in batches (n), after every n tests, DumpRenderTree is relaunched. "chunk" and "part" control which tests are run; sharding controls how they are run and is pretty much an internal detail that the user doesn't normally need to be aware of. (In reply to comment #7) > If a shard to DRT is almost always a directory, why not say: "running DRT in parallel over X directories" or something similar. Because it's not always a directory. Even in the default case, we carve all of the http tests into a few shards (1/4 the number of DRTs by default), which means that multiple directories are usually in a single shard here. Finally, I will note that the code refers to "shards" in multiple places. Any change should update the code as well as the text string in the message. I suggest that we either leave the wording alone, or make the sharding a debug message so that devs don't normally see it. I'm fine with either option. Learning a new use of a word is not a bad thing. I am also about to write more documentation about how NRWT works, and will need to refer to this concept as part of that (so it would be good to end this debate sooner rather than later). Perhaps having the documentation will partially address your concerns over confusing users not familiar with this usage. If you are dead set on wanting to change this, of all of the alternatives that have been suggested, "batch" is probably the second-best word to describe things after "shard". I don't think the --batch-size flag is used very often, and we could and should probably rename it to something more descriptive like --restart-drt-after-every-n-tests or something. I do not like any of those other suggestions that we use generic collective words like "part", "chunk", "group", or "section". There are already too many generic collections in NRWT and I have been working hard to get rid of them in order to make things clearer. Using one of these words would be a step backwards. 790751 9 rniwa 2012-12-13 10:44:13 -0800 (In reply to comment #8) > (In reply to comment #6) > > I agree that sometimes jargon can be good for its precision, but you have to consider the possibility that a particular term may confuse more than it enlightens. > > Of course. That's why I said I wasn't necessarily making that argument. > > > At least four WebKit contributors said on IRC that they actually found this confusing and misinterpreted the message. > > At least as many people on IRC said that they thought this was a correct and accurate term. And it may be that now that the others have had this explained, they are enlightened. I'm pretty sure that's how learning new vocabulary works. As far as I could recall, everyone who said they know what sharding mean in this context were all Google employees. > (In reply to comment #7) > Finally, I will note that the code refers to "shards" in multiple places. Any change should update the code as well as the text string in the message. > > I suggest that we either leave the wording alone, or make the sharding a debug message so that devs don't normally see it. I'm fine with either option. Making it a debug message will be fine with me so that ordinary users don't have to get confused by this message. > Learning a new use of a word is not a bad thing. As a user of this development tool, I don't want to learn or need to know anything new unless it's absolutely necessary to use the tool. The number of technical jargons we have, even in just NRWT, is absolutely insane. This whole elitism and egoism towards knowledge in our industry always drives me nuts. Times like this, I really consider stop being a software engineer and go do something else. And this is why I don't think software engineers should ever design UIs or write documentations. 790773 10 rniwa 2012-12-13 11:13:41 -0800 "test queues"? 791282 11 mjs 2012-12-13 20:35:19 -0800 (In reply to comment #8) > (In reply to comment #6) > > At least four WebKit contributors said on IRC that they actually found this confusing and misinterpreted the message. > > > > At least as many people on IRC said that they thought this was a correct and accurate term. And it may be that now that the others have had this explained, they are enlightened. I'm pretty sure that's how learning new vocabulary works. Come on, let's be a little less smug. This is a UI design problem, not an opportunity to show the class our superior knowledge of jargon terms. 791481 12 179453 rniwa 2012-12-14 02:07:54 -0800 Created attachment 179453 Only mention shards in the debug mode 791649 13 179453 dpranke 2012-12-14 09:06:20 -0800 Comment on attachment 179453 Only mention shards in the debug mode View in context: https://bugs.webkit.org/attachment.cgi?id=179453&action=review > Tools/Scripts/webkitpy/layout_tests/views/printing.py:119 > + self._print_debug("Running %d %ss in parallel over %d shards (%d locked)." % I would like to keep the "Running 5 DumpRenderTrees" part of the message showing up at the info level. That is a very useful message. 791727 14 179504 rniwa 2012-12-14 11:31:51 -0800 Created attachment 179504 Patch 791730 15 179504 dpranke 2012-12-14 11:34:48 -0800 Comment on attachment 179504 Patch Thanks. 791853 16 179504 webkit.review.bot 2012-12-14 14:14:56 -0800 Comment on attachment 179504 Patch Clearing flags on attachment: 179504 Committed r137770: <http://trac.webkit.org/changeset/137770> 791854 17 webkit.review.bot 2012-12-14 14:15:01 -0800 All reviewed patches have been landed. Closing bug. 179453 2012-12-14 02:07:54 -0800 2012-12-14 11:31:48 -0800 Only mention shards in the debug mode bug-104858-20121214020516.patch text/plain 1968 rniwa SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDEzNzczMykKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE3IEBACisyMDEyLTEyLTE0ICBSeW9zdWtlIE5pd2EgIDxybml3YUB3ZWJraXQub3JnPgor CisgICAgICAgICJSdW5uaW5nIDEgRHVtcFJlbmRlclRyZWUgb3ZlciBYIHNoYXJkcyIgaXMgbm90 IGEgaGVscGZ1bCBvdXRwdXQKKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19i dWcuY2dpP2lkPTEwNDg1OAorCisgICAgICAgIFJldmlld2VkIGJ5IE5PQk9EWSAoT09QUyEpLgor CisgICAgICAgIFRoZSB3b3JkICJzaGFyZCIgaXMgbm90IHR5cGljYWxseSB1c2VkIGluIHRlc3Qg ZnJhbWV3b3Jrcy4gVGhpcyB3b3JkIGlzIGNvbmZ1c2luZyBmb3IgbWFueSBXZWJLaXQKKyAgICAg ICAgY29udHJpYnV0b3JzIHdobyB0eXBpY2FsbHkgZG9uJ3QgaGF2ZSBtdWNoIGV4cGVyaWVuY2Ug b24gZGlzdHJpYnV0ZWQgY29tcHV0YXRpb25zIGFuZCBkYXRhYmFzZXMuCisgICAgICAgIFNvIG9u bHkgcHJpbnQgdGhpcyBtZXNzYWdlIG91dCBpbiB0aGUgZGVidWcgbW9kZS4KKworICAgICAgICAq IFNjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3ZpZXdzL3ByaW50aW5nLnB5OgorICAgICAg ICAoUHJpbnRlci5wcmludF93b3JrZXJzX2FuZF9zaGFyZHMpOgorCiAyMDEyLTEyLTEzICBFcmlj IFNlaWRlbCAgPGVyaWNAd2Via2l0Lm9yZz4KIAogICAgICAgICBBZGQgLS1wcm9maWxlcj1QUk9G SUxFUiBvcHRpb24gdG8gcnVuLXBlcmYtdGVzdHMgdG8gYWxsb3cgc3BlY2lmeWluZyB3aGljaCBw cm9maWxlciB0byB1c2Ugb24gcGxhdGZvcm1zIHdpdGggbWFueQpJbmRleDogVG9vbHMvU2NyaXB0 cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvdmlld3MvcHJpbnRpbmcucHkKPT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0g VG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvdmlld3MvcHJpbnRpbmcucHkJKHJl dmlzaW9uIDEzNzczMykKKysrIFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3Zp ZXdzL3ByaW50aW5nLnB5CSh3b3JraW5nIGNvcHkpCkBAIC0xMTQsOSArMTE0LDkgQEAgY2xhc3Mg UHJpbnRlcihvYmplY3QpOgogICAgIGRlZiBwcmludF93b3JrZXJzX2FuZF9zaGFyZHMoc2VsZiwg bnVtX3dvcmtlcnMsIG51bV9zaGFyZHMsIG51bV9sb2NrZWRfc2hhcmRzKToKICAgICAgICAgZHJp dmVyX25hbWUgPSBzZWxmLl9wb3J0LmRyaXZlcl9uYW1lKCkKICAgICAgICAgaWYgbnVtX3dvcmtl cnMgPT0gMToKLSAgICAgICAgICAgIHNlbGYuX3ByaW50X2RlZmF1bHQoIlJ1bm5pbmcgMSAlcyBv dmVyICVzLiIgJSAoZHJpdmVyX25hbWUsIGdyYW1tYXIucGx1cmFsaXplKCdzaGFyZCcsIG51bV9z aGFyZHMpKSkKKyAgICAgICAgICAgIHNlbGYuX3ByaW50X2RlYnVnKCJSdW5uaW5nIDEgJXMgb3Zl ciAlcy4iICUgKGRyaXZlcl9uYW1lLCBncmFtbWFyLnBsdXJhbGl6ZSgnc2hhcmQnLCBudW1fc2hh cmRzKSkpCiAgICAgICAgIGVsc2U6Ci0gICAgICAgICAgICBzZWxmLl9wcmludF9kZWZhdWx0KCJS dW5uaW5nICVkICVzcyBpbiBwYXJhbGxlbCBvdmVyICVkIHNoYXJkcyAoJWQgbG9ja2VkKS4iICUK KyAgICAgICAgICAgIHNlbGYuX3ByaW50X2RlYnVnKCJSdW5uaW5nICVkICVzcyBpbiBwYXJhbGxl bCBvdmVyICVkIHNoYXJkcyAoJWQgbG9ja2VkKS4iICUKICAgICAgICAgICAgICAgICAobnVtX3dv cmtlcnMsIGRyaXZlcl9uYW1lLCBudW1fc2hhcmRzLCBudW1fbG9ja2VkX3NoYXJkcykpCiAgICAg ICAgIHNlbGYuX3ByaW50X2RlZmF1bHQoJycpCiAK 179504 2012-12-14 11:31:51 -0800 2012-12-14 14:14:56 -0800 Patch bug-104858-20121214112911.patch text/plain 3242 rniwa SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDEzNzczMykKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE4IEBACisyMDEyLTEyLTE0ICBSeW9zdWtlIE5pd2EgIDxybml3YUB3ZWJraXQub3JnPgor CisgICAgICAgICJSdW5uaW5nIDEgRHVtcFJlbmRlclRyZWUgb3ZlciBYIHNoYXJkcyIgaXMgbm90 IGEgaGVscGZ1bCBvdXRwdXQKKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19i dWcuY2dpP2lkPTEwNDg1OAorCisgICAgICAgIFJldmlld2VkIGJ5IE5PQk9EWSAoT09QUyEpLgor CisgICAgICAgIFRoZSB3b3JkICJzaGFyZCIgaXMgbm90IHR5cGljYWxseSB1c2VkIGluIHRlc3Qg ZnJhbWV3b3Jrcy4gVGhpcyB3b3JkIGlzIGNvbmZ1c2luZyBmb3IgbWFueSBXZWJLaXQKKyAgICAg ICAgY29udHJpYnV0b3JzIHdobyB0eXBpY2FsbHkgZG9uJ3QgaGF2ZSBtdWNoIGV4cGVyaWVuY2Ug b24gZGlzdHJpYnV0ZWQgY29tcHV0YXRpb25zIGFuZCBkYXRhYmFzZXMuCisgICAgICAgIFNvIG9u bHkgcHJpbnQgdGhpcyBtZXNzYWdlIG91dCBpbiB0aGUgZGVidWcgbW9kZS4KKworICAgICAgICAq IFNjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3J1bl93ZWJraXRfdGVzdHNfaW50ZWdyYXRp b250ZXN0LnB5OgorICAgICAgICAqIFNjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3ZpZXdz L3ByaW50aW5nLnB5OgorICAgICAgICAoUHJpbnRlci5wcmludF93b3JrZXJzX2FuZF9zaGFyZHMp OgorCiAyMDEyLTEyLTEzICBFcmljIFNlaWRlbCAgPGVyaWNAd2Via2l0Lm9yZz4KIAogICAgICAg ICBBZGQgLS1wcm9maWxlcj1QUk9GSUxFUiBvcHRpb24gdG8gcnVuLXBlcmYtdGVzdHMgdG8gYWxs b3cgc3BlY2lmeWluZyB3aGljaCBwcm9maWxlciB0byB1c2Ugb24gcGxhdGZvcm1zIHdpdGggbWFu eQpJbmRleDogVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvcnVuX3dlYmtpdF90 ZXN0c19pbnRlZ3JhdGlvbnRlc3QucHkKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvU2NyaXB0cy93ZWJr aXRweS9sYXlvdXRfdGVzdHMvcnVuX3dlYmtpdF90ZXN0c19pbnRlZ3JhdGlvbnRlc3QucHkJKHJl dmlzaW9uIDEzNzczMykKKysrIFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3J1 bl93ZWJraXRfdGVzdHNfaW50ZWdyYXRpb250ZXN0LnB5CSh3b3JraW5nIGNvcHkpCkBAIC0zMTcs NyArMzE3LDcgQEAgY2xhc3MgTWFpblRlc3QodW5pdHRlc3QuVGVzdENhc2UsIFN0cmVhbQogICAg ICAgICAgICAgZGVsIG9zLmVudmlyb25bIldFQktJVF9URVNUX01BWF9MT0NLRURfU0hBUkRTIl0K ICAgICAgICAgXywgcmVndWxhcl9vdXRwdXQsIF8gPSBsb2dnaW5nX3J1bihbJy0tZGVidWctcnd0 LWxvZ2dpbmcnLCAnLS1jaGlsZC1wcm9jZXNzZXMnLCAnMiddLCBzaGFyZWRfcG9ydD1GYWxzZSkK ICAgICAgICAgdHJ5OgotICAgICAgICAgICAgc2VsZi5hc3NlcnRUcnVlKGFueShbJygxIGxvY2tl ZCknIGluIGxpbmUgZm9yIGxpbmUgaW4gcmVndWxhcl9vdXRwdXQuYnVmbGlzdF0pKQorICAgICAg ICAgICAgc2VsZi5hc3NlcnRUcnVlKGFueShbJzEgbG9ja2VkJyBpbiBsaW5lIGZvciBsaW5lIGlu IHJlZ3VsYXJfb3V0cHV0LmJ1Zmxpc3RdKSkKICAgICAgICAgZmluYWxseToKICAgICAgICAgICAg IGlmIHNhdmVfZW52X3dlYmtpdF90ZXN0X21heF9sb2NrZWRfc2hhcmRzOgogICAgICAgICAgICAg ICAgIG9zLmVudmlyb25bIldFQktJVF9URVNUX01BWF9MT0NLRURfU0hBUkRTIl0gPSBzYXZlX2Vu dl93ZWJraXRfdGVzdF9tYXhfbG9ja2VkX3NoYXJkcwpJbmRleDogVG9vbHMvU2NyaXB0cy93ZWJr aXRweS9sYXlvdXRfdGVzdHMvdmlld3MvcHJpbnRpbmcucHkKPT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMv U2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvdmlld3MvcHJpbnRpbmcucHkJKHJldmlzaW9u IDEzNzczMykKKysrIFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3ZpZXdzL3By aW50aW5nLnB5CSh3b3JraW5nIGNvcHkpCkBAIC0xMTQsMTAgKzExNCwxMSBAQCBjbGFzcyBQcmlu dGVyKG9iamVjdCk6CiAgICAgZGVmIHByaW50X3dvcmtlcnNfYW5kX3NoYXJkcyhzZWxmLCBudW1f d29ya2VycywgbnVtX3NoYXJkcywgbnVtX2xvY2tlZF9zaGFyZHMpOgogICAgICAgICBkcml2ZXJf bmFtZSA9IHNlbGYuX3BvcnQuZHJpdmVyX25hbWUoKQogICAgICAgICBpZiBudW1fd29ya2VycyA9 PSAxOgotICAgICAgICAgICAgc2VsZi5fcHJpbnRfZGVmYXVsdCgiUnVubmluZyAxICVzIG92ZXIg JXMuIiAlIChkcml2ZXJfbmFtZSwgZ3JhbW1hci5wbHVyYWxpemUoJ3NoYXJkJywgbnVtX3NoYXJk cykpKQorICAgICAgICAgICAgc2VsZi5fcHJpbnRfZGVmYXVsdCgiUnVubmluZyAxICVzLiIgJSBk cml2ZXJfbmFtZSkKKyAgICAgICAgICAgIHNlbGYuX3ByaW50X2RlYnVnKCIoJXMpLiIgJSBncmFt bWFyLnBsdXJhbGl6ZSgnc2hhcmQnLCBudW1fc2hhcmRzKSkKICAgICAgICAgZWxzZToKLSAgICAg ICAgICAgIHNlbGYuX3ByaW50X2RlZmF1bHQoIlJ1bm5pbmcgJWQgJXNzIGluIHBhcmFsbGVsIG92 ZXIgJWQgc2hhcmRzICglZCBsb2NrZWQpLiIgJQotICAgICAgICAgICAgICAgIChudW1fd29ya2Vy cywgZHJpdmVyX25hbWUsIG51bV9zaGFyZHMsIG51bV9sb2NrZWRfc2hhcmRzKSkKKyAgICAgICAg ICAgIHNlbGYuX3ByaW50X2RlZmF1bHQoIlJ1bm5pbmcgJWQgJXNzIGluIHBhcmFsbGVsLiIgJSAo bnVtX3dvcmtlcnMsIGRyaXZlcl9uYW1lKSkKKyAgICAgICAgICAgIHNlbGYuX3ByaW50X2RlYnVn KCIoJWQgc2hhcmRzOyAlZCBsb2NrZWQpLiIgJSAobnVtX3NoYXJkcywgbnVtX2xvY2tlZF9zaGFy ZHMpKQogICAgICAgICBzZWxmLl9wcmludF9kZWZhdWx0KCcnKQogCiAgICAgZGVmIF9wcmludF9l eHBlY3RlZF9yZXN1bHRzX29mX3R5cGUoc2VsZiwgcnVuX3Jlc3VsdHMsIHJlc3VsdF90eXBlLCBy ZXN1bHRfdHlwZV9zdHIsIHRlc3RzX3dpdGhfcmVzdWx0X3R5cGVfY2FsbGJhY2spOgo=