216705 2020-09-18 10:34:22 -0700 Make ews-app robust against config.json issues 2020-09-21 12:19:39 -0700 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED InRadar P2 Normal --- 1 aakash_jain aakash_jain aakash_jain ap jbedard ryanhaddad webkit-bug-importer oldest_to_newest 1689851 0 aakash_jain 2020-09-18 10:34:22 -0700 If config.json is invalid temporarily (e.g.: merge conflict while updating checkout on the buildbot server), then the ews-app might encounter a crash and stop polling for new patches. I noticed this on UAT instance. Noticed this exception in logs: 2020-09-17 20:51:17,020 - 22 r? patches, 0 patches need to be sent to Buildbot: [] 2020-09-17 20:51:47,050 - Fetching: https://ews-build.webkit-uat.org/config.json Exception in thread Thread-1: Traceback (most recent call last): File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner self.run() File "/usr/lib64/python2.7/threading.py", line 765, in run self.__target(*self.__args, **self.__kwargs) File "/var/ews/OpenSource/Tools/BuildSlaveSupport/ews-app/ews/fetcher.py", line 50, in run Buildbot.update_icons_for_queues_mapping() File "/var/ews/OpenSource/Tools/BuildSlaveSupport/ews-app/ews/common/buildbot.py", line 99, in update_icons_for_queues_mapping config = cls.fetch_config() File "/var/ews/OpenSource/Tools/BuildSlaveSupport/ews-app/ews/common/buildbot.py", line 95, in fetch_config return config_data.json() File "/usr/lib/python2.7/site-packages/requests/models.py", line 795, in json return json.loads(self.content.decode(encoding), **kwargs) File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode obj, end = self.scan_once(s, idx) ValueError: Expecting property name enclosed in double quotes: line 358 column 1 (char 6271) 1689857 1 409145 aakash_jain 2020-09-18 10:37:43 -0700 Created attachment 409145 Patch 1689879 2 409145 jbedard 2020-09-18 11:14:55 -0700 Comment on attachment 409145 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=409145&action=review > Tools/BuildSlaveSupport/ews-app/ews/common/buildbot.py:97 > + except Exception as e: Nit: Don't need the "as e" 1689899 3 409145 aakash_jain 2020-09-18 12:06:27 -0700 Comment on attachment 409145 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=409145&action=review >> Tools/BuildSlaveSupport/ews-app/ews/common/buildbot.py:97 >> + except Exception as e: > > Nit: Don't need the "as e" Updated the patch to add a log which includes the error as well. This log might help in debugging. 1689900 4 409159 aakash_jain 2020-09-18 12:06:39 -0700 Created attachment 409159 Patch 1689922 5 ews-feeder 2020-09-18 12:46:39 -0700 Committed r267277: <https://trac.webkit.org/changeset/267277> All reviewed patches have been landed. Closing bug and clearing flags on attachment 409159. 1689924 6 webkit-bug-importer 2020-09-18 12:47:23 -0700 <rdar://problem/69169472> 1690097 7 409159 ap 2020-09-19 11:22:04 -0700 Comment on attachment 409159 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=409159&action=review > Tools/BuildSlaveSupport/ews-app/ews/common/buildbot.py:105 > _log.warn('Unable to fetch buildbot config.json') Looks like we now call both _log.error and _log.warn. 1690423 8 409159 aakash_jain 2020-09-21 12:19:39 -0700 Comment on attachment 409159 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=409159&action=review >> Tools/BuildSlaveSupport/ews-app/ews/common/buildbot.py:105 >> _log.warn('Unable to fetch buildbot config.json') > > Looks like we now call both _log.error and _log.warn. Changed this one to _log.error as well in https://trac.webkit.org/changeset/267353/webkit 409145 2020-09-18 10:37:43 -0700 2020-09-18 12:06:36 -0700 Patch bug-216705-20200918133742.patch text/plain 1631 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI2NzI0OSkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE0IEBACisyMDIwLTA5LTE4ICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBNYWtlIGV3cy1hcHAgcm9idXN0IGFnYWluc3QgY29uZmlnLmpzb24gaXNz dWVzCisgICAgICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD0yMTY3 MDUKKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAgICAqIEJ1 aWxkU2xhdmVTdXBwb3J0L2V3cy1hcHAvZXdzL2NvbW1vbi9idWlsZGJvdC5weToKKyAgICAgICAg KEJ1aWxkYm90LmZldGNoX2NvbmZpZyk6CisgICAgICAgIChCdWlsZGJvdC51cGRhdGVfaWNvbnNf Zm9yX3F1ZXVlc19tYXBwaW5nKToKKwogMjAyMC0wOS0xOCAgRGllZ28gUGlubyBHYXJjaWEgIDxk cGlub0BpZ2FsaWEuY29tPgogCiAgICAgICAgIFtld3NdIFNldCBvbmx5IG9uZSB3b3JrZXIgZm9y IEpTQyBBUk12NyBidWlsZGVyIGFuZCB0ZXN0ZXIgcXVldWVzCkluZGV4OiBUb29scy9CdWlsZFNs YXZlU3VwcG9ydC9ld3MtYXBwL2V3cy9jb21tb24vYnVpbGRib3QucHkKPT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0g VG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvZXdzLWFwcC9ld3MvY29tbW9uL2J1aWxkYm90LnB5CShy ZXZpc2lvbiAyNjcyNDkpCisrKyBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9ld3MtYXBwL2V3cy9j b21tb24vYnVpbGRib3QucHkJKHdvcmtpbmcgY29weSkKQEAgLTkyLDEzICs5MiwxNyBAQCBjbGFz cyBCdWlsZGJvdCgpOgogICAgICAgICBjb25maWdfZGF0YSA9IHV0aWwuZmV0Y2hfZGF0YV9mcm9t X3VybChjb25maWdfdXJsKQogICAgICAgICBpZiBub3QgY29uZmlnX2RhdGE6CiAgICAgICAgICAg ICByZXR1cm4ge30KLSAgICAgICAgcmV0dXJuIGNvbmZpZ19kYXRhLmpzb24oKQorICAgICAgICB0 cnk6CisgICAgICAgICAgICByZXR1cm4gY29uZmlnX2RhdGEuanNvbigpCisgICAgICAgIGV4Y2Vw dCBFeGNlcHRpb24gYXMgZToKKyAgICAgICAgICAgIHJldHVybiB7fQogCiAgICAgQGNsYXNzbWV0 aG9kCiAgICAgZGVmIHVwZGF0ZV9pY29uc19mb3JfcXVldWVzX21hcHBpbmcoY2xzKToKICAgICAg ICAgY29uZmlnID0gY2xzLmZldGNoX2NvbmZpZygpCiAgICAgICAgIGlmIG5vdCBjb25maWc6CiAg ICAgICAgICAgICBfbG9nLndhcm4oJ1VuYWJsZSB0byBmZXRjaCBidWlsZGJvdCBjb25maWcuanNv bicpCisgICAgICAgICAgICByZXR1cm4KICAgICAgICAgZm9yIGJ1aWxkZXIgaW4gY29uZmlnLmdl dCgnYnVpbGRlcnMnLCBbXSk6CiAgICAgICAgICAgICBzaG9ydG5hbWUgPSBidWlsZGVyLmdldCgn c2hvcnRuYW1lJykKICAgICAgICAgICAgIEJ1aWxkYm90Lmljb25zX2Zvcl9xdWV1ZXNfbWFwcGlu Z1tzaG9ydG5hbWVdID0gYnVpbGRlci5nZXQoJ2ljb24nKQo= 409159 2020-09-18 12:06:39 -0700 2020-09-18 12:46:39 -0700 Patch bug-216705-20200918150638.patch text/plain 1713 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI2NzI0OSkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE0IEBACisyMDIwLTA5LTE4ICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBNYWtlIGV3cy1hcHAgcm9idXN0IGFnYWluc3QgY29uZmlnLmpzb24gaXNz dWVzCisgICAgICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD0yMTY3 MDUKKworICAgICAgICBSZXZpZXdlZCBieSBKb25hdGhhbiBCZWRhcmQuCisKKyAgICAgICAgKiBC dWlsZFNsYXZlU3VwcG9ydC9ld3MtYXBwL2V3cy9jb21tb24vYnVpbGRib3QucHk6CisgICAgICAg IChCdWlsZGJvdC5mZXRjaF9jb25maWcpOgorICAgICAgICAoQnVpbGRib3QudXBkYXRlX2ljb25z X2Zvcl9xdWV1ZXNfbWFwcGluZyk6CisKIDIwMjAtMDktMTggIERpZWdvIFBpbm8gR2FyY2lhICA8 ZHBpbm9AaWdhbGlhLmNvbT4KIAogICAgICAgICBbZXdzXSBTZXQgb25seSBvbmUgd29ya2VyIGZv ciBKU0MgQVJNdjcgYnVpbGRlciBhbmQgdGVzdGVyIHF1ZXVlcwpJbmRleDogVG9vbHMvQnVpbGRT bGF2ZVN1cHBvcnQvZXdzLWFwcC9ld3MvY29tbW9uL2J1aWxkYm90LnB5Cj09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0t IFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1hcHAvZXdzL2NvbW1vbi9idWlsZGJvdC5weQko cmV2aXNpb24gMjY3MjQ5KQorKysgVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvZXdzLWFwcC9ld3Mv Y29tbW9uL2J1aWxkYm90LnB5CSh3b3JraW5nIGNvcHkpCkBAIC05MiwxMyArOTIsMTggQEAgY2xh c3MgQnVpbGRib3QoKToKICAgICAgICAgY29uZmlnX2RhdGEgPSB1dGlsLmZldGNoX2RhdGFfZnJv bV91cmwoY29uZmlnX3VybCkKICAgICAgICAgaWYgbm90IGNvbmZpZ19kYXRhOgogICAgICAgICAg ICAgcmV0dXJuIHt9Ci0gICAgICAgIHJldHVybiBjb25maWdfZGF0YS5qc29uKCkKKyAgICAgICAg dHJ5OgorICAgICAgICAgICAgcmV0dXJuIGNvbmZpZ19kYXRhLmpzb24oKQorICAgICAgICBleGNl cHQgRXhjZXB0aW9uIGFzIGU6CisgICAgICAgICAgICBfbG9nLmVycm9yKCdFcnJvciBpbiBmZXRj aGluZyB7fS4gRXJyb3I6IHt9Jy5mb3JtYXQoY29uZmlnX3VybCwgZSkpCisgICAgICAgICAgICBy ZXR1cm4ge30KIAogICAgIEBjbGFzc21ldGhvZAogICAgIGRlZiB1cGRhdGVfaWNvbnNfZm9yX3F1 ZXVlc19tYXBwaW5nKGNscyk6CiAgICAgICAgIGNvbmZpZyA9IGNscy5mZXRjaF9jb25maWcoKQog ICAgICAgICBpZiBub3QgY29uZmlnOgogICAgICAgICAgICAgX2xvZy53YXJuKCdVbmFibGUgdG8g ZmV0Y2ggYnVpbGRib3QgY29uZmlnLmpzb24nKQorICAgICAgICAgICAgcmV0dXJuCiAgICAgICAg IGZvciBidWlsZGVyIGluIGNvbmZpZy5nZXQoJ2J1aWxkZXJzJywgW10pOgogICAgICAgICAgICAg c2hvcnRuYW1lID0gYnVpbGRlci5nZXQoJ3Nob3J0bmFtZScpCiAgICAgICAgICAgICBCdWlsZGJv dC5pY29uc19mb3JfcXVldWVzX21hcHBpbmdbc2hvcnRuYW1lXSA9IGJ1aWxkZXIuZ2V0KCdpY29u JykK