176853 2017-09-13 10:28:23 -0700 spindumps created on Mac/ios testers are very large 2022-09-07 08:28:17 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED InRadar P2 Normal --- 1 lforschler lforschler aakash_jain ap buildbot dean_johnson jbedard lforschler thisiskatewinslet webkit-bug-importer oldest_to_newest 1348338 0 lforschler 2017-09-13 10:28:23 -0700 Some Mac and iOS testers include spindumps. These are very large when extracted out to the results folder on build.webkit.org. (multiple terabytes) We should leave these compressed on disk. simply pre-compressing them on the client, before we create the results archive would be sufficient. 1348339 1 lforschler 2017-09-13 10:28:45 -0700 <rdar://problem/34397653> 1348340 2 320654 lforschler 2017-09-13 10:29:06 -0700 Created attachment 320654 v1 patch for review 1348341 3 buildbot 2017-09-13 10:30:43 -0700 Attachment 320654 did not pass style-queue: ERROR: Tools/ChangeLog:1: ChangeLog entry has no bug number [changelog/bugnumber] [5] Total errors found: 1 in 2 files If any of these errors are false positives, please file a bug against check-webkit-style. 1348345 4 320655 lforschler 2017-09-13 10:35:03 -0700 Created attachment 320655 correct ChangeLog to include the bugzilla entry 1348442 5 320655 aakash_jain 2017-09-13 12:48:59 -0700 Comment on attachment 320655 correct ChangeLog to include the bugzilla entry View in context: https://bugs.webkit.org/attachment.cgi?id=320655&action=review looks good assuming you have tested it. > Tools/BuildSlaveSupport/test-result-archive:50 > + shutil.copyfileobj(f_in, f_out) does this compress the file? 1348444 6 320655 lforschler 2017-09-13 12:51:28 -0700 Comment on attachment 320655 correct ChangeLog to include the bugzilla entry View in context: https://bugs.webkit.org/attachment.cgi?id=320655&action=review >> Tools/BuildSlaveSupport/test-result-archive:50 >> + shutil.copyfileobj(f_in, f_out) > > does this compress the file? Yes... this is straight from the python docs, so it seems to be the standard way of doing it. https://docs.python.org/2/library/gzip.html 1348446 7 lforschler 2017-09-13 12:54:22 -0700 Committed revision 221985. 320654 2017-09-13 10:29:06 -0700 2017-09-13 10:35:03 -0700 v1 patch for review compress_spindumps.patch text/plain 3159 lforschler SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDIyMTk3NCkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE3IEBACisyMDE3LTA5LTEzICBMdWNhcyBGb3JzY2hsZXIgIDxsZm9yc2NobGVyQGFwcGxl LmNvbT4KKworICAgICAgICBDb21wcmVzcyBzcGluZHVtcHMgb24gY2xpZW50IHNpZGUgYmVmb3Jl IHBhY2thZ2luZyBpbnRvIGxheW91dCB0ZXN0IHJlc3VsdHMgemlwIChtYWMgJiBpb3MpCisgICAg ICAgIDxyZGFyOi8vcHJvYmxlbS8zNDM5NzY1Mz4KKyAgICAgICAgCisgICAgICAgIFJldmlld2Vk IGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgICogQnVpbGRTbGF2ZVN1cHBvcnQvdGVzdC1y ZXN1bHQtYXJjaGl2ZToKKyAgICAgICAgKG1haW4pOgorICAgICAgICAoZ3ppcF9maWxlKTogdXNl IGd6aXAgdG8gY29tcHJlc3MgYSBmaWxlLCByZW1vdmluZyBvcmlnaW5hbAorICAgICAgICAoY29t cHJlc3Nfc3BpbmR1bXBzKTogZmluZCBhbmQgY29tcHJlc3Mgc3BpbmR1bXBzIG1hdGNoaW5nIGEg ZmlsdGVyCisgICAgICAgIChhcmNoaXZlX3Rlc3RfcmVzdWx0cyk6IHJlZmFjdG9yIHRvIHNuYWtl X2Nhc2UKKyAgICAgICAgKGFyY2hpdmVUZXN0UmVzdWx0cyk6IERlbGV0ZWQuCisKIDIwMTctMDkt MTIgIFdlbnNvbiBIc2llaCAgPHdlbnNvbl9oc2llaEBhcHBsZS5jb20+CiAKICAgICAgICAgW2lP UyBEbkRdIFN1cHBvcnQgRGF0YVRyYW5zZmVyLmdldERhdGEgYW5kIERhdGFUcmFuc2Zlci5zZXRE YXRhIHdoZW4gZHJhZ2dpbmcgb3IgZHJvcHBpbmcKSW5kZXg6IFRvb2xzL0J1aWxkU2xhdmVTdXBw b3J0L3Rlc3QtcmVzdWx0LWFyY2hpdmUKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvQnVpbGRTbGF2ZVN1 cHBvcnQvdGVzdC1yZXN1bHQtYXJjaGl2ZQkocmV2aXNpb24gMjIxOTc0KQorKysgVG9vbHMvQnVp bGRTbGF2ZVN1cHBvcnQvdGVzdC1yZXN1bHQtYXJjaGl2ZQkod29ya2luZyBjb3B5KQpAQCAtMjMs NyArMjMsNyBAQAogIyAoSU5DTFVESU5HIE5FR0xJR0VOQ0UgT1IgT1RIRVJXSVNFKSBBUklTSU5H IElOIEFOWSBXQVkgT1VUIE9GIFRIRSBVU0UgT0YKICMgVEhJUyBTT0ZUV0FSRSwgRVZFTiBJRiBB RFZJU0VEIE9GIFRIRSBQT1NTSUJJTElUWSBPRiBTVUNIIERBTUFHRS4KIAotaW1wb3J0IG9wdHBh cnNlLCBvcywgc2h1dGlsLCBzdWJwcm9jZXNzLCBzeXMKK2ltcG9ydCBnemlwLCBvcHRwYXJzZSwg b3MsIHNodXRpbCwgc3VicHJvY2Vzcywgc3lzCiAKIHNvdXJjZVJvb3REaXJlY3RvcnkgPSBvcy5w YXRoLmFic3BhdGgob3MucGF0aC5qb2luKG9zLnBhdGguZGlybmFtZShfX2ZpbGVfXyksICIuLiIs ICIuLiIpKQogYXJjaGl2ZUZpbGUgPSBvcy5wYXRoLmpvaW4oc291cmNlUm9vdERpcmVjdG9yeSwg ImxheW91dC10ZXN0LXJlc3VsdHMuemlwIikKQEAgLTQzLDkgKzQzLDIyIEBAIGRlZiBtYWluKCk6 CiAgICAgICAgIHBhcnNlci5lcnJvcigiQWN0aW9uIGlzIHJlcXVpcmVkIikKIAogICAgIGxheW91 dFRlc3RSZXN1bHRzRGlyID0gb3MucGF0aC5hYnNwYXRoKG9zLnBhdGguam9pbihzb3VyY2VSb290 RGlyZWN0b3J5LCAibGF5b3V0LXRlc3QtcmVzdWx0cyIpKQotICAgIHJldHVybiBhcmNoaXZlVGVz dFJlc3VsdHMob3B0aW9ucy5jb25maWd1cmF0aW9uLCBvcHRpb25zLnBsYXRmb3JtLCBsYXlvdXRU ZXN0UmVzdWx0c0RpcikKKyAgICByZXR1cm4gYXJjaGl2ZV90ZXN0X3Jlc3VsdHMob3B0aW9ucy5j b25maWd1cmF0aW9uLCBvcHRpb25zLnBsYXRmb3JtLCBsYXlvdXRUZXN0UmVzdWx0c0RpcikKIAot ZGVmIGFyY2hpdmVUZXN0UmVzdWx0cyhjb25maWd1cmF0aW9uLCBwbGF0Zm9ybSwgbGF5b3V0VGVz dFJlc3VsdHNEaXIpOgorZGVmIGd6aXBfZmlsZShyb290LCBuYW1lKToKKyAgICB3aXRoIG9wZW4o b3MucGF0aC5qb2luKHJvb3QsIG5hbWUpLCAncmInKSBhcyBmX2luLCBnemlwLm9wZW4ob3MucGF0 aC5qb2luKHJvb3QsIG5hbWUgKyAnLmd6JyksICd3YicpIGFzIGZfb3V0OgorICAgICAgICBzaHV0 aWwuY29weWZpbGVvYmooZl9pbiwgZl9vdXQpCisgICAgICAgIG9zLnJlbW92ZShvcy5wYXRoLmpv aW4ocm9vdCwgbmFtZSkpCisKK2RlZiBjb21wcmVzc19zcGluZHVtcHMobGF5b3V0VGVzdFJlc3Vs dHNEaXIpOgorICAgIGZpbGVfcGF0dGVybnMgPSAoJy1zcGluZHVtcC50eHQnLCctc2FtcGxlLnR4 dCcpCisgICAgZm9yIHJvb3QsIGRpcnMsIGZpbGVzIGluIG9zLndhbGsobGF5b3V0VGVzdFJlc3Vs dHNEaXIpOgorICAgICAgICBmb3IgbmFtZSBpbiBmaWxlczoKKyAgICAgICAgICAgIGZvciBwYXR0 ZXJuIGluIGZpbGVfcGF0dGVybnM6ICAgICAgICAKKyAgICAgICAgICAgICAgICBpZiBwYXR0ZXJu IGluIG5hbWU6CisgICAgICAgICAgICAgICAgICAgIGd6aXBfZmlsZShyb290LCBuYW1lKQorCitk ZWYgYXJjaGl2ZV90ZXN0X3Jlc3VsdHMoY29uZmlndXJhdGlvbiwgcGxhdGZvcm0sIGxheW91dFRl c3RSZXN1bHRzRGlyKToKICAgICBhc3NlcnQgcGxhdGZvcm0gaW4gKCdtYWMnLCAnd2luJywgJ2d0 aycsICd3aW5jYWlybycsICdpb3MnLCAnd3BlJykKIAogICAgIHRyeToKQEAgLTY0LDYgKzc3LDcg QEAgZGVmIGFyY2hpdmVUZXN0UmVzdWx0cyhjb25maWd1cmF0aW9uLCBwbAogICAgIG9wZW4ob3Mu cGF0aC5qb2luKGxheW91dFRlc3RSZXN1bHRzRGlyLCAnLnBsYWNlaG9sZGVyJyksICd3JykuY2xv c2UoKQogCiAgICAgaWYgcGxhdGZvcm0gaW4gKCdtYWMnLCAnaW9zJyk6CisgICAgICAgIGNvbXBy ZXNzX3NwaW5kdW1wcyhsYXlvdXRUZXN0UmVzdWx0c0RpcikKICAgICAgICAgaWYgc3VicHJvY2Vz cy5jYWxsKFsiZGl0dG8iLCAiLWMiLCAiLWsiLCAiLS1zZXF1ZXN0ZXJSc3JjIiwgIi0temxpYkNv bXByZXNzaW9uTGV2ZWwiLCAiMiIsIGxheW91dFRlc3RSZXN1bHRzRGlyLCBhcmNoaXZlRmlsZV0p OgogICAgICAgICAgICAgcmV0dXJuIDEKICAgICBlbGlmIHBsYXRmb3JtIGluICgnd2luJywgJ2d0 aycsICd3aW5jYWlybycsICd3cGUnKToK 320655 2017-09-13 10:35:03 -0700 2017-09-13 12:48:59 -0700 correct ChangeLog to include the bugzilla entry compress_spindumps.patch text/plain 3172 lforschler SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDIyMTk3NCkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE3IEBACisyMDE3LTA5LTEzICBMdWNhcyBGb3JzY2hsZXIgIDxsZm9yc2NobGVyQGFwcGxl LmNvbT4KKworICAgICAgICBDb21wcmVzcyBzcGluZHVtcHMgb24gY2xpZW50IHNpZGUgYmVmb3Jl IHBhY2thZ2luZyBpbnRvIGxheW91dCB0ZXN0IHJlc3VsdHMgemlwIChtYWMgJiBpb3MpCisgICAg ICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD0xNzY4NTMKKworICAg ICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAgICAqIEJ1aWxkU2xhdmVT dXBwb3J0L3Rlc3QtcmVzdWx0LWFyY2hpdmU6CisgICAgICAgIChtYWluKToKKyAgICAgICAgKGd6 aXBfZmlsZSk6IHVzZSBnemlwIHRvIGNvbXByZXNzIGEgZmlsZSwgcmVtb3Zpbmcgb3JpZ2luYWwK KyAgICAgICAgKGNvbXByZXNzX3NwaW5kdW1wcyk6IGZpbmQgYW5kIGNvbXByZXNzIHNwaW5kdW1w cyBtYXRjaGluZyBhIGZpbHRlcgorICAgICAgICAoYXJjaGl2ZV90ZXN0X3Jlc3VsdHMpOiByZWZh Y3RvciB0byBzbmFrZV9jYXNlCisgICAgICAgIChhcmNoaXZlVGVzdFJlc3VsdHMpOiBEZWxldGVk LgorCiAyMDE3LTA5LTEyICBXZW5zb24gSHNpZWggIDx3ZW5zb25faHNpZWhAYXBwbGUuY29tPgog CiAgICAgICAgIFtpT1MgRG5EXSBTdXBwb3J0IERhdGFUcmFuc2Zlci5nZXREYXRhIGFuZCBEYXRh VHJhbnNmZXIuc2V0RGF0YSB3aGVuIGRyYWdnaW5nIG9yIGRyb3BwaW5nCkluZGV4OiBUb29scy9C dWlsZFNsYXZlU3VwcG9ydC90ZXN0LXJlc3VsdC1hcmNoaXZlCj09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIFRvb2xz L0J1aWxkU2xhdmVTdXBwb3J0L3Rlc3QtcmVzdWx0LWFyY2hpdmUJKHJldmlzaW9uIDIyMTk3NCkK KysrIFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L3Rlc3QtcmVzdWx0LWFyY2hpdmUJKHdvcmtpbmcg Y29weSkKQEAgLTIzLDcgKzIzLDcgQEAKICMgKElOQ0xVRElORyBORUdMSUdFTkNFIE9SIE9USEVS V0lTRSkgQVJJU0lORyBJTiBBTlkgV0FZIE9VVCBPRiBUSEUgVVNFIE9GCiAjIFRISVMgU09GVFdB UkUsIEVWRU4gSUYgQURWSVNFRCBPRiBUSEUgUE9TU0lCSUxJVFkgT0YgU1VDSCBEQU1BR0UuCiAK LWltcG9ydCBvcHRwYXJzZSwgb3MsIHNodXRpbCwgc3VicHJvY2Vzcywgc3lzCitpbXBvcnQgZ3pp cCwgb3B0cGFyc2UsIG9zLCBzaHV0aWwsIHN1YnByb2Nlc3MsIHN5cwogCiBzb3VyY2VSb290RGly ZWN0b3J5ID0gb3MucGF0aC5hYnNwYXRoKG9zLnBhdGguam9pbihvcy5wYXRoLmRpcm5hbWUoX19m aWxlX18pLCAiLi4iLCAiLi4iKSkKIGFyY2hpdmVGaWxlID0gb3MucGF0aC5qb2luKHNvdXJjZVJv b3REaXJlY3RvcnksICJsYXlvdXQtdGVzdC1yZXN1bHRzLnppcCIpCkBAIC00Myw5ICs0MywyMiBA QCBkZWYgbWFpbigpOgogICAgICAgICBwYXJzZXIuZXJyb3IoIkFjdGlvbiBpcyByZXF1aXJlZCIp CiAKICAgICBsYXlvdXRUZXN0UmVzdWx0c0RpciA9IG9zLnBhdGguYWJzcGF0aChvcy5wYXRoLmpv aW4oc291cmNlUm9vdERpcmVjdG9yeSwgImxheW91dC10ZXN0LXJlc3VsdHMiKSkKLSAgICByZXR1 cm4gYXJjaGl2ZVRlc3RSZXN1bHRzKG9wdGlvbnMuY29uZmlndXJhdGlvbiwgb3B0aW9ucy5wbGF0 Zm9ybSwgbGF5b3V0VGVzdFJlc3VsdHNEaXIpCisgICAgcmV0dXJuIGFyY2hpdmVfdGVzdF9yZXN1 bHRzKG9wdGlvbnMuY29uZmlndXJhdGlvbiwgb3B0aW9ucy5wbGF0Zm9ybSwgbGF5b3V0VGVzdFJl c3VsdHNEaXIpCiAKLWRlZiBhcmNoaXZlVGVzdFJlc3VsdHMoY29uZmlndXJhdGlvbiwgcGxhdGZv cm0sIGxheW91dFRlc3RSZXN1bHRzRGlyKToKK2RlZiBnemlwX2ZpbGUocm9vdCwgbmFtZSk6Cisg ICAgd2l0aCBvcGVuKG9zLnBhdGguam9pbihyb290LCBuYW1lKSwgJ3JiJykgYXMgZl9pbiwgZ3pp cC5vcGVuKG9zLnBhdGguam9pbihyb290LCBuYW1lICsgJy5neicpLCAnd2InKSBhcyBmX291dDoK KyAgICAgICAgc2h1dGlsLmNvcHlmaWxlb2JqKGZfaW4sIGZfb3V0KQorICAgICAgICBvcy5yZW1v dmUob3MucGF0aC5qb2luKHJvb3QsIG5hbWUpKQorCitkZWYgY29tcHJlc3Nfc3BpbmR1bXBzKGxh eW91dFRlc3RSZXN1bHRzRGlyKToKKyAgICBmaWxlX3BhdHRlcm5zID0gKCctc3BpbmR1bXAudHh0 JywnLXNhbXBsZS50eHQnKQorICAgIGZvciByb290LCBkaXJzLCBmaWxlcyBpbiBvcy53YWxrKGxh eW91dFRlc3RSZXN1bHRzRGlyKToKKyAgICAgICAgZm9yIG5hbWUgaW4gZmlsZXM6CisgICAgICAg ICAgICBmb3IgcGF0dGVybiBpbiBmaWxlX3BhdHRlcm5zOiAgICAgICAgCisgICAgICAgICAgICAg ICAgaWYgcGF0dGVybiBpbiBuYW1lOgorICAgICAgICAgICAgICAgICAgICBnemlwX2ZpbGUocm9v dCwgbmFtZSkKKworZGVmIGFyY2hpdmVfdGVzdF9yZXN1bHRzKGNvbmZpZ3VyYXRpb24sIHBsYXRm b3JtLCBsYXlvdXRUZXN0UmVzdWx0c0Rpcik6CiAgICAgYXNzZXJ0IHBsYXRmb3JtIGluICgnbWFj JywgJ3dpbicsICdndGsnLCAnd2luY2Fpcm8nLCAnaW9zJywgJ3dwZScpCiAKICAgICB0cnk6CkBA IC02NCw2ICs3Nyw3IEBAIGRlZiBhcmNoaXZlVGVzdFJlc3VsdHMoY29uZmlndXJhdGlvbiwgcGwK ICAgICBvcGVuKG9zLnBhdGguam9pbihsYXlvdXRUZXN0UmVzdWx0c0RpciwgJy5wbGFjZWhvbGRl cicpLCAndycpLmNsb3NlKCkKIAogICAgIGlmIHBsYXRmb3JtIGluICgnbWFjJywgJ2lvcycpOgor ICAgICAgICBjb21wcmVzc19zcGluZHVtcHMobGF5b3V0VGVzdFJlc3VsdHNEaXIpCiAgICAgICAg IGlmIHN1YnByb2Nlc3MuY2FsbChbImRpdHRvIiwgIi1jIiwgIi1rIiwgIi0tc2VxdWVzdGVyUnNy YyIsICItLXpsaWJDb21wcmVzc2lvbkxldmVsIiwgIjIiLCBsYXlvdXRUZXN0UmVzdWx0c0Rpciwg YXJjaGl2ZUZpbGVdKToKICAgICAgICAgICAgIHJldHVybiAxCiAgICAgZWxpZiBwbGF0Zm9ybSBp biAoJ3dpbicsICdndGsnLCAnd2luY2Fpcm8nLCAnd3BlJyk6Cg==