215330 2020-08-10 12:21:27 -0700 [ews] Handle logs with unicode characters in ews emails 2020-08-18 14:37:25 -0700 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=215174 https://bugs.webkit.org/show_bug.cgi?id=215627 InRadar P2 Normal --- 1 aakash_jain aakash_jain aakash_jain ap darin dewei_zhu jbedard ryanhaddad webkit-bug-importer oldest_to_newest 1679142 0 aakash_jain 2020-08-10 12:21:27 -0700 We added relevant log information in ews emails in r265302. However, logs with unicode characters results in following exception. We should handle them properly. Traceback (most recent call last): File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 1475, in gotResult _inlineCallbacks(r, g, status) File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 1421, in _inlineCallbacks status.deferred.callback(getattr(e, "value", None)) File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 460, in callback self._startRunCallbacks(result) File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks self._runCallbacks() --- <exception caught here> --- File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks current.result = callback(current.result, *args, **kw) File "steps.py", line 1401, in <lambda> d.addCallback(lambda res: self.analyzeResults()) File "steps.py", line 1422, in analyzeResults self.send_email_for_new_build_failure() File "steps.py", line 1492, in send_email_for_new_build_failure email_text += '\n\nError lines:\n\n{}'.format(logs) exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in position 108: ordinal not in range(128) 1679146 1 406318 aakash_jain 2020-08-10 12:24:46 -0700 Created attachment 406318 Patch 1679154 2 aakash_jain 2020-08-10 12:36:04 -0700 Committed r265442: <https://trac.webkit.org/changeset/265442> 1679155 3 webkit-bug-importer 2020-08-10 12:37:56 -0700 <rdar://problem/66794326> 1679159 4 406318 dewei_zhu 2020-08-10 12:41:55 -0700 Comment on attachment 406318 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=406318&action=review > Tools/BuildSlaveSupport/ews-build/send_email.py:43 > + email_data = """From: {}\nTo: {}\nSubject: {}\n\n{}""".format(FROM_EMAIL, ', '.join(to_emails), subject, text.encode('utf-8')) Any reason that we don't do it for `subject`? 1679167 5 aakash_jain 2020-08-10 13:08:41 -0700 (In reply to dewei_zhu from comment #4) > Any reason that we don't do it for `subject`? Good point. Let me do that in a separate patch. 1679423 6 406318 jbedard 2020-08-11 09:21:30 -0700 Comment on attachment 406318 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=406318&action=review >> Tools/BuildSlaveSupport/ews-build/send_email.py:43 >> + email_data = """From: {}\nTo: {}\nSubject: {}\n\n{}""".format(FROM_EMAIL, ', '.join(to_emails), subject, text.encode('utf-8')) > > Any reason that we don't do it for `subject`? Fine for now, but. this may be a case for using webkitcorepy, because this encoding will be unneeded in Python 3. 406318 2020-08-10 12:24:46 -0700 2020-08-10 12:30:01 -0700 Patch bug-215330-20200810152445.patch text/plain 3196 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI2NTQzOCkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE2IEBACisyMDIwLTA4LTEwICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBbZXdzXSBIYW5kbGUgbG9ncyB3aXRoIHVuaWNvZGUgY2hhcmFjdGVycyBp biBld3MgZW1haWxzCisgICAgICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNn aT9pZD0yMTUzMzAKKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAg ICAgICAqIEJ1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zZW5kX2VtYWlsLnB5OgorICAgICAg ICAoc2VuZF9lbWFpbCk6IEVuY29kZSBlbWFpbCBjb250ZW50IGFzIHV0Zi04LgorICAgICAgICAq IEJ1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwcy5weToKKyAgICAgICAgKEFuYWx5emVD b21waWxlV2ViS2l0UmVzdWx0cy5zZW5kX2VtYWlsX2Zvcl9uZXdfYnVpbGRfZmFpbHVyZSk6Cisg ICAgICAgIChBbmFseXplQ29tcGlsZVdlYktpdFJlc3VsdHMuc2VuZF9lbWFpbF9mb3JfcHJlZXhp c3RpbmdfYnVpbGRfZmFpbHVyZSk6CisKIDIwMjAtMDgtMDkgIFdlbnNvbiBIc2llaCAgPHdlbnNv bl9oc2llaEBhcHBsZS5jb20+CiAKICAgICAgICAgUkVHUkVTU0lPTiAocjI2MDgzMSk6IFdlYiBw cm9jZXNzIGNyYXNoZXMgdW5kZXIgRWRpdG9yOjpzZXRDb21wb3NpdGlvbigpIGFmdGVyIG5hdmln YXRpbmcgd2l0aCBtYXJrZWQgdGV4dApJbmRleDogVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvZXdz LWJ1aWxkL3NlbmRfZW1haWwucHkKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvQnVpbGRTbGF2ZVN1cHBv cnQvZXdzLWJ1aWxkL3NlbmRfZW1haWwucHkJKHJldmlzaW9uIDI2NTQzOCkKKysrIFRvb2xzL0J1 aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zZW5kX2VtYWlsLnB5CSh3b3JraW5nIGNvcHkpCkBA IC00MCw3ICs0MCw3IEBAIGRlZiBzZW5kX2VtYWlsKHRvX2VtYWlscywgc3ViamVjdCwgdGV4dCkK ICAgICAgICAgcHJpbnQoJ1NraXBwaW5nIGVtYWlsIHNpbmNlIG5vIHN1YmplY3Qgb3IgdGV4dCBp cyBzcGVjaWZpZWQnKQogICAgICAgICByZXR1cm4KIAotICAgIGVtYWlsX2RhdGEgPSAiIiJGcm9t OiB7fVxuVG86IHt9XG5TdWJqZWN0OiB7fVxuXG57fSIiIi5mb3JtYXQoRlJPTV9FTUFJTCwgJywg Jy5qb2luKHRvX2VtYWlscyksIHN1YmplY3QsIHRleHQpCisgICAgZW1haWxfZGF0YSA9ICIiIkZy b206IHt9XG5Ubzoge31cblN1YmplY3Q6IHt9XG5cbnt9IiIiLmZvcm1hdChGUk9NX0VNQUlMLCAn LCAnLmpvaW4odG9fZW1haWxzKSwgc3ViamVjdCwgdGV4dC5lbmNvZGUoJ3V0Zi04JykpCiAKICAg ICBzZXJ2ZXIgPSBzbXRwbGliLlNNVFAoU0VSVkVSKQogICAgIHNlcnZlci5zZW5kbWFpbChGUk9N X0VNQUlMLCB0b19lbWFpbHMsIGVtYWlsX2RhdGEpCkluZGV4OiBUb29scy9CdWlsZFNsYXZlU3Vw cG9ydC9ld3MtYnVpbGQvc3RlcHMucHkKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvQnVpbGRTbGF2ZVN1 cHBvcnQvZXdzLWJ1aWxkL3N0ZXBzLnB5CShyZXZpc2lvbiAyNjU0MzgpCisrKyBUb29scy9CdWls ZFNsYXZlU3VwcG9ydC9ld3MtYnVpbGQvc3RlcHMucHkJKHdvcmtpbmcgY29weSkKQEAgLTE0NzQs NyArMTQ3NCw3IEBAIGNsYXNzIEFuYWx5emVDb21waWxlV2ViS2l0UmVzdWx0cyhidWlsZHMKICAg ICAgICAgICAgIGVtYWlsX3RleHQgPSAnRVdTIGhhcyBkZXRlY3RlZCBidWlsZCBmYWlsdXJlIG9u IHt9IHdoaWxlIHRlc3RpbmcgUGF0Y2gge30uJy5mb3JtYXQoYnVpbGRlcl9uYW1lLCBwYXRjaF9p ZCkKICAgICAgICAgICAgIGVtYWlsX3RleHQgKz0gJ1xuXG5GdWxsIGRldGFpbHMgYXJlIGF2YWls YWJsZSBhdDoge31cblxuUGF0Y2ggYXV0aG9yOiB7fScuZm9ybWF0KGJ1aWxkX3VybCwgcGF0Y2hf YXV0aG9yKQogICAgICAgICAgICAgaWYgbG9nczoKLSAgICAgICAgICAgICAgICBlbWFpbF90ZXh0 ICs9ICdcblxuRXJyb3IgbGluZXM6XG5cbnt9Jy5mb3JtYXQobG9ncykKKyAgICAgICAgICAgICAg ICBlbWFpbF90ZXh0ICs9IHUnXG5cbkVycm9yIGxpbmVzOlxuXG57fScuZm9ybWF0KGxvZ3MpCiAg ICAgICAgICAgICBlbWFpbF90ZXh0ICs9ICdcblxuVG8gdW5zdWJzY3JpYmxlIGZyb20gdGhlc2Ug bm90aWZpY2F0aW9ucyBvciB0byBwcm92aWRlIGFueSBmZWVkYmFjayBwbGVhc2UgZW1haWwgYWFr YXNoX2phaW5AYXBwbGUuY29tJwogICAgICAgICAgICAgc2VuZF9lbWFpbChbcGF0Y2hfYXV0aG9y XSwgZW1haWxfc3ViamVjdCwgZW1haWxfdGV4dCkKICAgICAgICAgZXhjZXB0IEV4Y2VwdGlvbiBh cyBlOgpAQCAtMTQ5NSw3ICsxNDk1LDcgQEAgY2xhc3MgQW5hbHl6ZUNvbXBpbGVXZWJLaXRSZXN1 bHRzKGJ1aWxkcwogICAgICAgICAgICAgZW1haWxfc3ViamVjdCA9ICdCdWlsZCBmYWlsdXJlIG9u IHRydW5rIG9uIHt9Jy5mb3JtYXQoYnVpbGRlcl9uYW1lKQogICAgICAgICAgICAgZW1haWxfdGV4 dCA9ICdGYWlsZWQgdG8gYnVpbGQgV2ViS2l0IHdpdGhvdXQgcGF0Y2ggaW4ge31cblxuQnVpbGRl cjoge31cblxuV29ya2VyOiB7fScuZm9ybWF0KGJ1aWxkX3VybCwgYnVpbGRlcl9uYW1lLCB3b3Jr ZXJfbmFtZSkKICAgICAgICAgICAgIGlmIGxvZ3M6Ci0gICAgICAgICAgICAgICAgZW1haWxfdGV4 dCArPSAnXG5cbkVycm9yIGxpbmVzOlxuXG57fScuZm9ybWF0KGxvZ3MpCisgICAgICAgICAgICAg ICAgZW1haWxfdGV4dCArPSB1J1xuXG5FcnJvciBsaW5lczpcblxue30nLmZvcm1hdChsb2dzKQog ICAgICAgICAgICAgc2VuZF9lbWFpbF90b19ib3Rfd2F0Y2hlcnMoZW1haWxfc3ViamVjdCwgZW1h aWxfdGV4dCkKICAgICAgICAgZXhjZXB0IEV4Y2VwdGlvbiBhcyBlOgogICAgICAgICAgICAgcHJp bnQoJ0Vycm9yIGluIHNlbmRpbmcgZW1haWwgZm9yIGJ1aWxkIGZhaWx1cmU6IHt9Jy5mb3JtYXQo ZSkpCg==