211781 2020-05-12 08:46:54 -0700 [Flatpak SDK] Craft a custom sccache config file from SDK toolchains 2020-05-13 01:59:16 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Local Build Unspecified Unspecified RESOLVED FIXED InRadar P2 Normal --- 1 pnormand pnormand clopez emilio ews-watchlist glenn jbedard psaavedra webkit-bug-importer oldest_to_newest 1651741 0 pnormand 2020-05-12 08:46:54 -0700 `webkit-flatpak -r -t <token>` would generate a config file in WebKitBuild/UserFlatpak/. This would be optional though. A global user config file (from ~/.config/sccache/) can still be used if the generated config doesn't exist. 1651744 1 399128 pnormand 2020-05-12 08:50:49 -0700 Created attachment 399128 Patch 1651757 2 399128 psaavedra 2020-05-12 09:13:54 -0700 Comment on attachment 399128 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=399128&action=review > Tools/flatpak/flatpakutils.py:873 > + sccache_config = {'dist': {'scheduler_url': 'https://sccache.igalia.com', What about?: `sccache_config = {'dist': {'scheduler_url': sccache_scheduler,` with: `general.add_argument("-s", "--sccache-scheduler", dest="sccache_scheduler", default='https://sccache.igalia.com')` 1651778 3 399128 clopez 2020-05-12 09:48:08 -0700 Comment on attachment 399128 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=399128&action=review > Tools/flatpak/flatpakutils.py:878 > + with open(self.sccache_config_file, 'w') as config: > + sccache_config = {'dist': {'scheduler_url': 'https://sccache.igalia.com', > + 'toolchain_cache_size': 5368709120, > + 'auth': {'type': 'token', > + 'token': self.sccache_token}, > + 'toolchains': toolchains}} > + toml.dump(sccache_config, config) To me it feels overkill to add a new dependency (toml) to just be able to write the sccache config in json/dict format instead of writing it directly in its own native toml format. Its there any reason to not write the config file directly in its own native format (avoiding this dependency)? You can use a templated variable to make that more pretty. Example: sccache_config_template = """\ [dist] scheduler_url = "%(scheduler_url)s" toolchain_cache_size = 5368709120 toolchains = "%(toolchains)s" [dist.auth] token = "%(sccache_token)s" type = "token" """ print (sccache_config_template % { "scheduler_url" : "https://sccache.igalia.com", "toolchains" : toolchains, "sccache_token" : self.sccache_token} ) 1651782 4 399128 pnormand 2020-05-12 09:58:13 -0700 Comment on attachment 399128 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=399128&action=review >> Tools/flatpak/flatpakutils.py:878 >> + toml.dump(sccache_config, config) > > To me it feels overkill to add a new dependency (toml) to just be able to write the sccache config in json/dict format instead of writing it directly in its own native toml format. > Its there any reason to not write the config file directly in its own native format (avoiding this dependency)? > You can use a templated variable to make that more pretty. Example: > > sccache_config_template = """\ > [dist] > scheduler_url = "%(scheduler_url)s" > toolchain_cache_size = 5368709120 > toolchains = "%(toolchains)s" > > [dist.auth] > token = "%(sccache_token)s" > type = "token" > """ > > > print (sccache_config_template % { > "scheduler_url" : "https://sccache.igalia.com", > "toolchains" : toolchains, > "sccache_token" : self.sccache_token} ) You'd need a template for the toolchains too. My solution looks simpler imho :) And toml is a small python package anyway, automatically installed. I don't buy the overkill argument :) 1651788 5 clopez 2020-05-12 10:13:40 -0700 (In reply to Philippe Normand from comment #4) > Comment on attachment 399128 [details] > Patch > > View in context: > https://bugs.webkit.org/attachment.cgi?id=399128&action=review > > >> Tools/flatpak/flatpakutils.py:878 > >> + toml.dump(sccache_config, config) > > > > To me it feels overkill to add a new dependency (toml) to just be able to write the sccache config in json/dict format instead of writing it directly in its own native toml format. > > Its there any reason to not write the config file directly in its own native > > You'd need a template for the toolchains too. Right.. i missed that > My solution looks simpler imho > :) And toml is a small python package anyway, automatically installed. I > don't buy the overkill argument :) ok, fair enough 1651789 6 399128 clopez 2020-05-12 10:15:20 -0700 Comment on attachment 399128 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=399128&action=review >> Tools/flatpak/flatpakutils.py:873 >> + sccache_config = {'dist': {'scheduler_url': 'https://sccache.igalia.com', > > What about?: > > `sccache_config = {'dist': {'scheduler_url': sccache_scheduler,` > > with: > > `general.add_argument("-s", "--sccache-scheduler", dest="sccache_scheduler", default='https://sccache.igalia.com')` I like this suggestion of making the url configurable > Tools/flatpak/flatpakutils.py:874 > + 'toolchain_cache_size': 5368709120, The default seems to be 10GB. Why lowering it to 5GB? 1651790 7 psaavedra 2020-05-12 10:18:16 -0700 (In reply to Philippe Normand from comment #4) > Comment on attachment 399128 [details] > Patch > > View in context: > https://bugs.webkit.org/attachment.cgi?id=399128&action=review > > >> Tools/flatpak/flatpakutils.py:878 > >> + toml.dump(sccache_config, config) > > > > To me it feels overkill to add a new dependency (toml) to just be able to write the sccache config in json/dict format instead of writing it directly in its own native toml format. > > Its there any reason to not write the config file directly in its own native format (avoiding this dependency)? > > You can use a templated variable to make that more pretty. Example: > > > > sccache_config_template = """\ > > [dist] > > scheduler_url = "%(scheduler_url)s" > > toolchain_cache_size = 5368709120 > > toolchains = "%(toolchains)s" > > > > [dist.auth] > > token = "%(sccache_token)s" > > type = "token" > > """ > > > > > > print (sccache_config_template % { > > "scheduler_url" : "https://sccache.igalia.com", > > "toolchains" : toolchains, > > "sccache_token" : self.sccache_token} ) > > You'd need a template for the toolchains too. My solution looks simpler imho > :) And toml is a small python package anyway, automatically installed. I > don't buy the overkill argument :) Just a simple comment is not configparser enough to built this particular file template? 1651792 8 psaavedra 2020-05-12 10:19:17 -0700 *build 1651796 9 psaavedra 2020-05-12 10:26:15 -0700 (In reply to Pablo Saavedra from comment #7) > (In reply to Philippe Normand from comment #4) > > Comment on attachment 399128 [details] > > Patch > > > > View in context: > > https://bugs.webkit.org/attachment.cgi?id=399128&action=review > > > > >> Tools/flatpak/flatpakutils.py:878 > > >> + toml.dump(sccache_config, config) > > > > > > To me it feels overkill to add a new dependency (toml) to just be able to write the sccache config in json/dict format instead of writing it directly in its own native toml format. > > > Its there any reason to not write the config file directly in its own native format (avoiding this dependency)? > > > You can use a templated variable to make that more pretty. Example: > > > > > > sccache_config_template = """\ > > > [dist] > > > scheduler_url = "%(scheduler_url)s" > > > toolchain_cache_size = 5368709120 > > > toolchains = "%(toolchains)s" > > > > > > [dist.auth] > > > token = "%(sccache_token)s" > > > type = "token" > > > """ > > > > > > > > > print (sccache_config_template % { > > > "scheduler_url" : "https://sccache.igalia.com", > > > "toolchains" : toolchains, > > > "sccache_token" : self.sccache_token} ) > > > > You'd need a template for the toolchains too. My solution looks simpler imho > > :) And toml is a small python package anyway, automatically installed. I > > don't buy the overkill argument :) > > Just a simple comment is not configparser enough to built this particular > file template? Nevermind. I see the resulting file differs too much than a regular INI file: [dist] scheduler_url = "https://sccache.igalia.com" toolchain_cache_size = 5368709120 [[dist.toolchains]] compiler_executable = "/usr/bin/c++" archive_compiler_executable = "/usr/bin/g++" type = "path_override" archive = "/home/psaavedra/local/git/webkit-gtk/WebKit/WebKitBuild/Toolchains/webkit-sdk-gcc-c636916ef47c8b977cbfab46ae5f3289.tar.gz" [[dist.toolchains]] compiler_executable = "/usr/bin/clang++" archive_compiler_executable = "/usr/bin/clang++" type = "path_override" archive = "/home/psaavedra/local/git/webkit-gtk/WebKit/WebKitBuild/Toolchains/webkit-sdk-clang-a0c3c4672a324be2250371fbe01c10fb.tar.gz" [dist.auth] token = "XXXXXXXXXXXXXXXXXXX" type = "token" 1652154 10 pnormand 2020-05-13 01:43:37 -0700 (In reply to Carlos Alberto Lopez Perez from comment #6) > > I like this suggestion of making the url configurable > Ok, let's do it then. > > Tools/flatpak/flatpakutils.py:874 > > + 'toolchain_cache_size': 5368709120, > > The default seems to be 10GB. Why lowering it to 5GB? No reason, I'll remove this :) 1652156 11 pnormand 2020-05-13 01:58:15 -0700 Committed r261607: <https://trac.webkit.org/changeset/261607> 1652157 12 webkit-bug-importer 2020-05-13 01:59:16 -0700 <rdar://problem/63175727> 399128 2020-05-12 08:50:49 -0700 2020-05-12 10:15:20 -0700 Patch bug-211781-20200512165047.patch text/plain 8931 pnormand U3VidmVyc2lvbiBSZXZpc2lvbjogMjYxNTUzCmRpZmYgLS1naXQgYS9Ub29scy9DaGFuZ2VMb2cg Yi9Ub29scy9DaGFuZ2VMb2cKaW5kZXggYjlmYTNiZWI0NTExNDRhYjAxOTAzYzg2ZDQ3MDhkZjU1 ZGIyNWU5My4uODA2YmM4NjcyYWI3NmU3NGQ1YmZmMDFlYjYwY2U1ZWNlNDRjNzNkNCAxMDA2NDQK LS0tIGEvVG9vbHMvQ2hhbmdlTG9nCisrKyBiL1Rvb2xzL0NoYW5nZUxvZwpAQCAtMSwzICsxLDI4 IEBACisyMDIwLTA1LTEyICBQaGlsaXBwZSBOb3JtYW5kICA8cG5vcm1hbmRAaWdhbGlhLmNvbT4K KworICAgICAgICBbRmxhdHBhayBTREtdIENyYWZ0IGEgY3VzdG9tIHNjY2FjaGUgY29uZmlnIGZp bGUgZnJvbSBTREsgdG9vbGNoYWlucworICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9z aG93X2J1Zy5jZ2k/aWQ9MjExNzgxCisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BT ISkuCisKKyAgICAgICAgT3B0aW9uYWxseSBnZW5lcmF0ZSBhIHNjY2FjaGUgY29uZmlnIGZpbGUg YW5kIHVzZSBpdCB3aGVuIGJ1aWxkaW5nIFdlYktpdC4gRm9yIG5vdyB0aGUKKyAgICAgICAgc2Nj YWNoZS5pZ2FsaWEuY29tIHNjaGVkdWxlciBpcyBoYXJkY29kZWQsIHRoaXMgaXMgbm90IGlkZWFs LiBTdWdnZXN0aW9ucyB3ZWxjb21lIDopCisKKyAgICAgICAgKiBTY3JpcHRzL3dlYmtpdHB5L3Ro aXJkcGFydHkvX19pbml0X18ucHk6CisgICAgICAgIChBdXRvaW5zdGFsbEltcG9ydEhvb2suZmlu ZF9tb2R1bGUpOgorICAgICAgICAoQXV0b2luc3RhbGxJbXBvcnRIb29rLl9pbnN0YWxsX3RvbWwp OgorICAgICAgICAqIGZsYXRwYWsvZmxhdHBha3V0aWxzLnB5OgorICAgICAgICAoV2Via2l0Rmxh dHBhay5sb2FkX2Zyb21fYXJncyk6CisgICAgICAgIChXZWJraXRGbGF0cGFrLl9faW5pdF9fKToK KyAgICAgICAgKFdlYmtpdEZsYXRwYWsuY2xlYW5fYXJncyk6CisgICAgICAgIChXZWJraXRGbGF0 cGFrLmhvc3RfcGF0aF90b19zYW5kYm94X3BhdGgpOgorICAgICAgICAoV2Via2l0RmxhdHBhay5y dW5faW5fc2FuZGJveCk6CisgICAgICAgIChXZWJraXRGbGF0cGFrLm1haW4pOgorICAgICAgICAo V2Via2l0RmxhdHBhay5zYXZlX2NvbmZpZyk6CisgICAgICAgIChXZWJraXRGbGF0cGFrKToKKyAg ICAgICAgKFdlYmtpdEZsYXRwYWsucGFja190b29sY2hhaW4pOgorICAgICAgICAoV2Via2l0Rmxh dHBhay5zZXR1cF9pY2VjYyk6IERlbGV0ZWQuCisKIDIwMjAtMDUtMTIgIENhcmxvcyBBbGJlcnRv IExvcGV6IFBlcmV6ICA8Y2xvcGV6QGlnYWxpYS5jb20+CiAKICAgICAgICAgV1BUIHRlc3QgaW1w b3J0ZXIgZmFpbHMgdG8gdXBkYXRlIHJlZmVyZW5jZXMgaW4gPGxpbms+IHRhZ3MKZGlmZiAtLWdp dCBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvdGhpcmRwYXJ0eS9fX2luaXRfXy5weSBiL1Rvb2xz L1NjcmlwdHMvd2Via2l0cHkvdGhpcmRwYXJ0eS9fX2luaXRfXy5weQppbmRleCAxOTA2NDU1OWQy Y2VmMTI4ZDliNDI1NjQwNDdjMGQ4MmYyYWRhNzAxLi5kZTY4NzQ3OTFkNzQxMDE5OWJlMDgyNTYw OTg1YTNiNTNlYTIxOTE1IDEwMDY0NAotLS0gYS9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L3RoaXJk cGFydHkvX19pbml0X18ucHkKKysrIGIvVG9vbHMvU2NyaXB0cy93ZWJraXRweS90aGlyZHBhcnR5 L19faW5pdF9fLnB5CkBAIC0xMjgsNiArMTI4LDggQEAgY2xhc3MgQXV0b2luc3RhbGxJbXBvcnRI b29rKG9iamVjdCk6CiAgICAgICAgICAgICBzZWxmLl9pbnN0YWxsX2JlYXV0aWZ1bHNvdXAoKQog ICAgICAgICBlbGlmICcuaHRtbDVsaWInIGluIGZ1bGxuYW1lOgogICAgICAgICAgICAgc2VsZi5f aW5zdGFsbF9odG1sNWxpYigpCisgICAgICAgIGVsaWYgJy50b21sJyBpbiBmdWxsbmFtZToKKyAg ICAgICAgICAgIHNlbGYuX2luc3RhbGxfdG9tbCgpCiAKICAgICBkZWYgX2luc3RhbGxfc2l4KHNl bGYpOgogICAgICAgICBzZWxmLl9pbnN0YWxsKCJodHRwczovL2ZpbGVzLnB5dGhvbmhvc3RlZC5v cmcvcGFja2FnZXMvMTYvZDgvYmM2MzE2Y2Y5ODQxOTcxOWJkNTljOTE3NDIxOTRjMTExYjZmMmU4 NWFiYWM4OGU0OTZhZGVmYWY3YWZlL3NpeC0xLjExLjAudGFyLmd6IiwKQEAgLTMyMyw2ICszMjUs MTEgQEAgY2xhc3MgQXV0b2luc3RhbGxJbXBvcnRIb29rKG9iamVjdCk6CiAgICAgICAgICAgICBm dWxsX2ZpcmVmb3hfdXJsID0gcmUuZmluZGFsbChyJy4qJXMuKicgJSBmaWxlbmFtZV9wb3N0Zml4 LCBhbGxfZmlyZWZveF9yZWxlYXNlX3VybHMpWzBdLnNwbGl0KCciJylbM10KICAgICAgICAgICAg IHNlbGYuaW5zdGFsbF9iaW5hcnkoZnVsbF9maXJlZm94X3VybCwgJ2dlY2tvZHJpdmVyJykKIAor ICAgIGRlZiBfaW5zdGFsbF90b21sKHNlbGYpOgorICAgICAgICB0b21sX2RpciA9IHNlbGYuX2Zz LmpvaW4oX0FVVE9JTlNUQUxMRURfRElSLCAidG9tbCIpCisgICAgICAgIGluc3RhbGxlciA9IEF1 dG9JbnN0YWxsZXIocHJlcGVuZF90b19zZWFyY2hfcGF0aD1UcnVlLCB0YXJnZXRfZGlyPXRvbWxf ZGlyKQorICAgICAgICBpbnN0YWxsZXIuaW5zdGFsbCh1cmw9Imh0dHBzOi8vZmlsZXMucHl0aG9u aG9zdGVkLm9yZy9wYWNrYWdlcy9iOS8xOS81Y2JkNzhlYWM4YjE3ODM2NzFjNDBlMzRiYjBmYTgz MTMzYTA2ZDM0MGEzOGI1NWM2NDUwNzZkNDAwOTQvdG9tbC0wLjEwLjAudGFyLmd6IiwgdXJsX3N1 YnBhdGg9InRvbWwtMC4xMC4wL3RvbWwiKQorCiAgICAgZGVmIF9pbnN0YWxsKHNlbGYsIHVybCwg dXJsX3N1YnBhdGg9Tm9uZSwgdGFyZ2V0X25hbWU9Tm9uZSk6CiAgICAgICAgIGluc3RhbGxlciA9 IEF1dG9JbnN0YWxsZXIodGFyZ2V0X2Rpcj1fQVVUT0lOU1RBTExFRF9ESVIpCiAgICAgICAgIHJl dHVybiBpbnN0YWxsZXIuaW5zdGFsbCh1cmw9dXJsLCB1cmxfc3VicGF0aD11cmxfc3VicGF0aCwg dGFyZ2V0X25hbWU9dGFyZ2V0X25hbWUpCmRpZmYgLS1naXQgYS9Ub29scy9mbGF0cGFrL2ZsYXRw YWt1dGlscy5weSBiL1Rvb2xzL2ZsYXRwYWsvZmxhdHBha3V0aWxzLnB5CmluZGV4IGE2YTY0NjQw MjhkNGU4ZmM5NGY5ZDY4NWU4MzQ5ODFmYTQyNjllMGMuLjFjYTdlNmQxYmQ1M2ZiM2MyODcxOWI4 MThjZTg3NjgwNjYxYjRlMjAgMTAwNjQ0Ci0tLSBhL1Rvb2xzL2ZsYXRwYWsvZmxhdHBha3V0aWxz LnB5CisrKyBiL1Rvb2xzL2ZsYXRwYWsvZmxhdHBha3V0aWxzLnB5CkBAIC0zNyw2ICszNyw4IEBA IGltcG9ydCByZQogZnJvbSB3ZWJraXRweS5jb21tb24uc3lzdGVtLnN5c3RlbWhvc3QgaW1wb3J0 IFN5c3RlbUhvc3QKIGZyb20gd2Via2l0cHkucG9ydC5mYWN0b3J5IGltcG9ydCBQb3J0RmFjdG9y eQogZnJvbSB3ZWJraXRweS5jb21tb24uc3lzdGVtLmxvZ3V0aWxzIGltcG9ydCBjb25maWd1cmVf bG9nZ2luZworaW1wb3J0IHdlYmtpdHB5LnRoaXJkcGFydHkuYXV0b2luc3RhbGxlZC50b21sCitp bXBvcnQgdG9tbAogCiB0cnk6CiAgICAgZnJvbSB1cmxsaWIucGFyc2UgaW1wb3J0IHVybHBhcnNl ICAjIHB5bGludDogZGlzYWJsZT1FMDYxMQpAQCAtNDIwLDYgKzQyMiw3IEBAIGNsYXNzIFdlYmtp dEZsYXRwYWs6CiAgICAgICAgIGdlbmVyYWwuYWRkX2FyZ3VtZW50KCItLXVzZS1pY2VjcmVhbSIs IGRlc3Q9InVzZV9pY2VjcmVhbSIsIGhlbHA9IlVzZSB0aGUgZGlzdHJpYnV0ZWQgaWNlY3JlYW0g KGljZWNjKSBjb21waWxlci4iLCBhY3Rpb249InN0b3JlX3RydWUiKQogICAgICAgICBnZW5lcmFs LmFkZF9hcmd1bWVudCgiLXIiLCAiLS1yZWdlbmVyYXRlLXRvb2xjaGFpbnMiLCBkZXN0PSJyZWdl bmVyYXRlX3Rvb2xjaGFpbnMiLCBhY3Rpb249InN0b3JlX3RydWUiLAogICAgICAgICAgICAgICAg ICAgICAgICAgICAgICBoZWxwPSJSZWdlbmVyYXRlIEljZUNDIGRpc3RyaWJ1YWJsZSB0b29sY2hh aW4gYXJjaGl2ZXMiKQorICAgICAgICBnZW5lcmFsLmFkZF9hcmd1bWVudCgiLXQiLCAiLS1zY2Nh Y2hlLXRva2VuIiwgZGVzdD0ic2NjYWNoZV90b2tlbiIpCiAgICAgICAgIGdlbmVyYWwuYWRkX2Fy Z3VtZW50KCItLXJlcG8iLCBoZWxwPSJGaWxlc3lzdGVtIGFic29sdXRlIHBhdGggdG8gdGhlIEZs YXRwYWsgcmVwb3NpdG9yeSB0byB1c2UiLCBkZXN0PSJ1c2VyX3JlcG8iKQogCiAgICAgICAgIGRl YnVnb3B0aW9ucyA9IHBhcnNlci5hZGRfYXJndW1lbnRfZ3JvdXAoIkRlYnVnZ2luZyIpCkBAIC00 NzUsNiArNDc4LDcgQEAgY2xhc3MgV2Via2l0RmxhdHBhazoKICAgICAgICAgc2VsZi51c2VfaWNl Y3JlYW0gPSBGYWxzZQogICAgICAgICBzZWxmLmljY192ZXJzaW9uID0ge30KICAgICAgICAgc2Vs Zi5yZWdlbmVyYXRlX3Rvb2xjaGFpbnMgPSBGYWxzZQorICAgICAgICBzZWxmLnNjY2FjaGVfdG9r ZW4gPSAiIgogCiAgICAgZGVmIGV4ZWN1dGVfY29tbWFuZChzZWxmLCBhcmdzLCBzdGRvdXQ9Tm9u ZSwgc3RkZXJyPU5vbmUpOgogICAgICAgICBfbG9nLmRlYnVnKCdSdW5uaW5nIGluIHNhbmRib3g6 ICVzXG4nICUgJyAnLmpvaW4oYXJncykpCkBAIC01MjAsNiArNTI0LDcgQEAgY2xhc3MgV2Via2l0 RmxhdHBhazoKICAgICAgICAgc2VsZi5idWlsZF9yb290ID0gb3MucGF0aC5qb2luKHNlbGYuc291 cmNlX3Jvb3QsICdXZWJLaXRCdWlsZCcpCiAgICAgICAgIHNlbGYuYnVpbGRfcGF0aCA9IG9zLnBh dGguam9pbihzZWxmLmJ1aWxkX3Jvb3QsIHNlbGYucGxhdGZvcm0sIHNlbGYuYnVpbGRfdHlwZSkK ICAgICAgICAgc2VsZi5jb25maWdfZmlsZSA9IG9zLnBhdGguam9pbihzZWxmLmZsYXRwYWtfYnVp bGRfcGF0aCwgJ3dlYmtpdF9mbGF0cGFrX2NvbmZpZy5qc29uJykKKyAgICAgICAgc2VsZi5zY2Nh Y2hlX2NvbmZpZ19maWxlID0gb3MucGF0aC5qb2luKHNlbGYuZmxhdHBha19idWlsZF9wYXRoLCAn c2NjYWNoZS50b21sJykKIAogICAgICAgICBDb25zb2xlLnF1aWV0ID0gc2VsZi5xdWlldAogICAg ICAgICBpZiBub3QgY2hlY2tfZmxhdHBhaygpOgpAQCAtNjA3LDYgKzYxMiwxMCBAQCBjbGFzcyBX ZWJraXRGbGF0cGFrOgogICAgICAgICAgICAgICAgIHJldHVybiBUcnVlCiAgICAgICAgIHJldHVy biBGYWxzZQogCisgICAgZGVmIGhvc3RfcGF0aF90b19zYW5kYm94X3BhdGgoc2VsZiwgaG9zdF9w YXRoKToKKyAgICAgICAgIyBGb3Igbm93IHRoaXMgc3VwcG9ydHMgb25seSBmaWxlcyBpbiB0aGUg V2ViS2l0IHBhdGgKKyAgICAgICAgcmV0dXJuIGhvc3RfcGF0aC5yZXBsYWNlKHNlbGYuc291cmNl X3Jvb3QsIHNlbGYuc2FuZGJveF9zb3VyY2Vfcm9vdCkKKwogICAgIGRlZiBydW5faW5fc2FuZGJv eChzZWxmLCAqYXJncywgKiprd2FyZ3MpOgogICAgICAgICBzZWxmLnNldHVwX2J1aWxkZGlyKHN0 ZG91dD1rd2FyZ3MuZ2V0KCJzdGRvdXQiLCBzeXMuc3Rkb3V0KSkKICAgICAgICAgY3dkID0ga3dh cmdzLnBvcCgiY3dkIiwgTm9uZSkKQEAgLTc2Miw2ICs3NzEsOSBAQCBjbGFzcyBXZWJraXRGbGF0 cGFrOgogICAgICAgICAgICAgX2xvZy5kZWJ1ZygiRW5hYmxpbmcgbmV0d29yayBhY2Nlc3MgZm9y IHRoZSByZW1vdGUgc2NjYWNoZSIpCiAgICAgICAgICAgICBmbGF0cGFrX2NvbW1hbmQuYXBwZW5k KHNoYXJlX25ldHdvcmtfb3B0aW9uKQogCisgICAgICAgICAgICBpZiBvcy5wYXRoLmlzZmlsZShz ZWxmLnNjY2FjaGVfY29uZmlnX2ZpbGUpIGFuZCBub3Qgc2VsZi5yZWdlbmVyYXRlX3Rvb2xjaGFp bnMgYW5kICJTQ0NBQ0hFX0NPTkYiIG5vdCBpbiBvcy5lbnZpcm9uLmtleXMoKToKKyAgICAgICAg ICAgICAgICBzYW5kYm94X2Vudmlyb25tZW50WyJTQ0NBQ0hFX0NPTkYiXSA9IHNlbGYuaG9zdF9w YXRoX3RvX3NhbmRib3hfcGF0aChzZWxmLnNjY2FjaGVfY29uZmlnX2ZpbGUpCisKICAgICAgICAg b3ZlcnJpZGVfc2NjYWNoZV9zZXJ2ZXJfcG9ydCA9IG9zLmVudmlyb24uZ2V0KCJXRUJLSVRfU0ND QUNIRV9TRVJWRVJfUE9SVCIpCiAgICAgICAgIGlmIG92ZXJyaWRlX3NjY2FjaGVfc2VydmVyX3Bv cnQ6CiAgICAgICAgICAgICBfbG9nLmRlYnVnKCJPdmVycmlkaW5nIHNjY2FjaGUgc2VydmVyIHBv cnQgdG8gJXMiICUgb3ZlcnJpZGVfc2NjYWNoZV9zZXJ2ZXJfcG9ydCkKQEAgLTgzMCw5ICs4NDIs OSBAQCBjbGFzcyBXZWJraXRGbGF0cGFrOgogCiAgICAgICAgIGlmIHJlZ2VuZXJhdGVfdG9vbGNo YWluczoKICAgICAgICAgICAgIHNlbGYuaWNjX3ZlcnNpb24gPSB7fQotICAgICAgICAgICAgc2Vs Zi5zZXR1cF9pY2VjYygiZ2NjIiwgImcrKyIpCi0gICAgICAgICAgICBzZWxmLnNldHVwX2ljZWNj KCJjbGFuZyIsICJjbGFuZysrIikKLSAgICAgICAgICAgIHNlbGYuc2F2ZV9jb25maWcoKQorICAg ICAgICAgICAgdG9vbGNoYWlucyA9IHNlbGYucGFja190b29sY2hhaW4oKCJnY2MiLCAiZysrIiks IHsiL3Vzci9iaW4vYysrIjogIi91c3IvYmluL2crKyJ9KQorICAgICAgICAgICAgdG9vbGNoYWlu cy5leHRlbmQoc2VsZi5wYWNrX3Rvb2xjaGFpbigoImNsYW5nIiwgImNsYW5nKysiKSwgeyIvdXNy L2Jpbi9jbGFuZysrIjogIi91c3IvYmluL2NsYW5nKysifSkpCisgICAgICAgICAgICBzZWxmLnNh dmVfY29uZmlnKHRvb2xjaGFpbnMpCiAKICAgICAgICAgcmV0dXJuIHNlbGYuc2V0dXBfZGV2X2Vu digpCiAKQEAgLTg0OCwxMiArODYwLDI1IEBAIGNsYXNzIFdlYmtpdEZsYXRwYWs6CiAgICAgZGVm IGhhc19lbnZpcm9ubWVudChzZWxmKToKICAgICAgICAgcmV0dXJuIG9zLnBhdGguZXhpc3RzKHNl bGYuZmxhdHBha19idWlsZF9wYXRoKQogCi0gICAgZGVmIHNhdmVfY29uZmlnKHNlbGYpOgorICAg IGRlZiBzYXZlX2NvbmZpZyhzZWxmLCB0b29sY2hhaW5zKToKICAgICAgICAgd2l0aCBvcGVuKHNl bGYuY29uZmlnX2ZpbGUsICd3JykgYXMgY29uZmlnOgogICAgICAgICAgICAganNvbl9jb25maWcg PSB7J2ljZWNjX3ZlcnNpb24nOiBzZWxmLmljY192ZXJzaW9ufQogICAgICAgICAgICAganNvbi5k dW1wKGpzb25fY29uZmlnLCBjb25maWcpCiAKLSAgICBkZWYgc2V0dXBfaWNlY2Moc2VsZiwgKmNv bXBpbGVycyk6CisgICAgICAgIGlmIG5vdCBzZWxmLnNjY2FjaGVfdG9rZW46CisgICAgICAgICAg ICBDb25zb2xlLm1lc3NhZ2UoIk5vIGF1dGhlbnRpY2F0aW9uIHRva2VuIHByb3ZpZGVkLiBSZS1y dW4gdGhpcyB3aXRoIHRoZSAtdCBvcHRpb24gaWYgYW4gc2NjYWNoZSB0b2tlbiB3YXMgcHJvdmlk ZWQgdG8geW91LiBTa2lwcGluZyBzY2NhY2hlIGNvbmZpZ3VyYXRpb24gZm9yIG5vdy4iKQorICAg ICAgICAgICAgcmV0dXJuCisKKyAgICAgICAgd2l0aCBvcGVuKHNlbGYuc2NjYWNoZV9jb25maWdf ZmlsZSwgJ3cnKSBhcyBjb25maWc6CisgICAgICAgICAgICBzY2NhY2hlX2NvbmZpZyA9IHsnZGlz dCc6IHsnc2NoZWR1bGVyX3VybCc6ICdodHRwczovL3NjY2FjaGUuaWdhbGlhLmNvbScsCisgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAndG9vbGNoYWluX2NhY2hlX3NpemUn OiA1MzY4NzA5MTIwLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgJ2F1 dGgnOiB7J3R5cGUnOiAndG9rZW4nLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgJ3Rva2VuJzogc2VsZi5zY2NhY2hlX3Rva2VufSwKKyAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICd0b29sY2hhaW5zJzogdG9vbGNoYWluc319Cisg ICAgICAgICAgICB0b21sLmR1bXAoc2NjYWNoZV9jb25maWcsIGNvbmZpZykKKyAgICAgICAgICAg IENvbnNvbGUubWVzc2FnZSgiQ3JlYXRlZCAlcyBzY2NhY2hlIGNvbmZpZyBmaWxlLiBJdCB3aWxs IGF1dG9tYXRpY2FsbHkgYmUgdXNlZCB3aGVuIGJ1aWxkaW5nIFdlYktpdCIsIHNlbGYuc2NjYWNo ZV9jb25maWdfZmlsZSkKKworICAgIGRlZiBwYWNrX3Rvb2xjaGFpbihzZWxmLCBjb21waWxlcnMs IHBhdGhfbWFwcGluZyk6CiAgICAgICAgIHdpdGggdGVtcGZpbGUuTmFtZWRUZW1wb3JhcnlGaWxl KCkgYXMgdG1wZmlsZToKICAgICAgICAgICAgIGNvbW1hbmQgPSBbJ2ljZWNjJywgJy0tYnVpbGQt bmF0aXZlJ10KICAgICAgICAgICAgIGNvbW1hbmQuZXh0ZW5kKFsiL3Vzci9iaW4vJXMiICUgY29t cGlsZXIgZm9yIGNvbXBpbGVyIGluIGNvbXBpbGVyc10pCkBAIC04NjksNiArODk0LDE1IEBAIGNs YXNzIFdlYmtpdEZsYXRwYWs6CiAgICAgICAgICAgICBzZWxmLmljY192ZXJzaW9uW2NvbXBpbGVy c1swXV0gPSBhcmNoaXZlX2ZpbGVuYW1lCiAgICAgICAgICAgICBDb25zb2xlLm1lc3NhZ2UoIkNy ZWF0ZWQgJXMgc2VsZi1jb250YWluZWQgdG9vbGNoYWluIGFyY2hpdmUiLCBhcmNoaXZlX2ZpbGVu YW1lKQogCisgICAgICAgICAgICBzY2NhY2hlX3Rvb2xjaGFpbnMgPSBbXQorICAgICAgICAgICAg Zm9yIChjb21waWxlcl9leGVjdXRhYmxlLCBhcmNoaXZlX2NvbXBpbGVyX2V4ZWN1dGFibGUpIGlu IHBhdGhfbWFwcGluZy5pdGVyaXRlbXMoKToKKyAgICAgICAgICAgICAgICBpdGVtID0geyd0eXBl JzogJ3BhdGhfb3ZlcnJpZGUnLAorICAgICAgICAgICAgICAgICAgICAgICAgJ2NvbXBpbGVyX2V4 ZWN1dGFibGUnOiBjb21waWxlcl9leGVjdXRhYmxlLAorICAgICAgICAgICAgICAgICAgICAgICAg J2FyY2hpdmUnOiBhcmNoaXZlX2ZpbGVuYW1lLAorICAgICAgICAgICAgICAgICAgICAgICAgJ2Fy Y2hpdmVfY29tcGlsZXJfZXhlY3V0YWJsZSc6IGFyY2hpdmVfY29tcGlsZXJfZXhlY3V0YWJsZX0K KyAgICAgICAgICAgICAgICBzY2NhY2hlX3Rvb2xjaGFpbnMuYXBwZW5kKGl0ZW0pCisgICAgICAg ICAgICByZXR1cm4gc2NjYWNoZV90b29sY2hhaW5zCisKICAgICBkZWYgc2V0dXBfZGV2X2Vudihz ZWxmKToKICAgICAgICAgaWYgbm90IG9zLnBhdGguZXhpc3RzKG9zLnBhdGguam9pbihzZWxmLmZs YXRwYWtfYnVpbGRfcGF0aCwgInJ1bnRpbWUiLCAib3JnLndlYmtpdC5TZGsiKSkgb3Igc2VsZi51 cGRhdGU6CiAgICAgICAgICAgICBzZWxmLmluc3RhbGxfYWxsKCkK