170158 2017-03-27 19:40:59 -0700 Fix dashboard test results on the dashboard 2017-03-28 18:31:35 -0700 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=168994 https://bugs.webkit.org/show_bug.cgi?id=170167 https://bugs.webkit.org/show_bug.cgi?id=170191 https://bugs.webkit.org/show_bug.cgi?id=170193 P2 Normal --- 1 ap ap aakash_jain commit-queue jmarcell lforschler ryanhaddad thorton webkit oldest_to_newest 1291752 0 ap 2017-03-27 19:40:59 -0700 - El Capitan test results are currently yellow, because dashboard tests fail there. - Link to dashboard test results points to stdio, which is not useful for these. 1291764 1 jmarcell 2017-03-27 20:09:52 -0700 The following commits broke the botwatcher's dashboard unit tests. https://bugs.webkit.org/show_bug.cgi?id=156595 https://bugs.webkit.org/show_bug.cgi?id=157680 I'm working with the authors of these commits to resolve the unit tests. You can run the tests in a headless terminal sessions via WebKitTestRunner as follows: run-dashboard-tests You can also run the tests in Safari: open ./Tools/BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/tests/index.html 1291766 2 jmarcell 2017-03-27 20:12:02 -0700 (In reply to Alexey Proskuryakov from comment #0) > - Link to dashboard test results points to stdio, which is not useful for > these. I followed the same convention as the layout tests. There is a link that shows a diff of expected vs. actual. 1291769 3 jmarcell 2017-03-27 20:13:59 -0700 (In reply to Jason Marcell from comment #2) > (In reply to Alexey Proskuryakov from comment #0) > > - Link to dashboard test results points to stdio, which is not useful for > > these. > > I followed the same convention as the layout tests. There is a link that > shows a diff of expected vs. actual. e.g. https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20(Tests)/r214433%20(174)/dashboard-layout-test-results/results.html 1291778 4 305554 ap 2017-03-27 20:31:27 -0700 Created attachment 305554 proposed fix 1291780 5 ap 2017-03-27 20:34:36 -0700 > I'm working with the authors of these commits to resolve the unit tests. Do you expect that this will happen soon (tomorrow)? If not, we should hide dashboard test results, so that the dashboard is not constantly red. 1291783 6 jmarcell 2017-03-27 20:39:23 -0700 (In reply to Alexey Proskuryakov from comment #5) > > I'm working with the authors of these commits to resolve the unit tests. > > Do you expect that this will happen soon (tomorrow)? If not, we should hide > dashboard test results, so that the dashboard is not constantly red. I doubt it will be fixed tomorrow. Hiding the results sounds fine. I didn't even know we could do that! 1291786 7 ap 2017-03-27 20:46:35 -0700 I don't think that there is existing support for hiding a step, we would need to add a temporary hack to the dashboard. Or it may be easier to mark the step as "hidden" in buildbot itself - the dashboard already checks for that. Would you be willing to look into that? I'm unlikely to have another opportunity to look at the code until Wednesday at best. 1291787 8 jmarcell 2017-03-27 20:51:00 -0700 (In reply to Alexey Proskuryakov from comment #7) > I don't think that there is existing support for hiding a step, we would > need to add a temporary hack to the dashboard. Or it may be easier to mark > the step as "hidden" in buildbot itself - the dashboard already checks for > that. > > Would you be willing to look into that? I'm unlikely to have another > opportunity to look at the code until Wednesday at best. Yep. No problem. 1292053 9 aakash_jain 2017-03-28 13:23:24 -0700 (In reply to Alexey Proskuryakov from comment #5) > Do you expect that this will happen soon (tomorrow)? I have fixed all the dashboard tests, pending review(170191 and 170193). 1292189 10 jmarcell 2017-03-28 16:51:00 -0700 I think we need to update Tools/BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/tests/index-expected.txt 1292201 11 jmarcell 2017-03-28 17:05:01 -0700 https://bugs.webkit.org/show_bug.cgi?id=170214 1292250 12 305554 commit-queue 2017-03-28 18:31:32 -0700 Comment on attachment 305554 proposed fix Clearing flags on attachment: 305554 Committed r214512: <http://trac.webkit.org/changeset/214512> 1292251 13 commit-queue 2017-03-28 18:31:35 -0700 All reviewed patches have been landed. Closing bug. 305554 2017-03-27 20:31:27 -0700 2017-03-28 18:31:32 -0700 proposed fix DashboardTests.txt text/plain 5656 ap SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDIxNDQ1NSkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDI1IEBACisyMDE3LTAzLTI3ICBBbGV4ZXkgUHJvc2t1cnlha292ICA8YXBAYXBwbGUuY29t PgorCisgICAgICAgIEZpeCBkYXNoYm9hcmQgdGVzdCByZXN1bHRzIG9uIHRoZSBkYXNoYm9hcmQK KyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTE3MDE1OAor CisgICAgICAgIFJldmlld2VkIGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgIExpbmsgdG8g cmVzdWx0IGRpZmYgaW5zdGVhZCBvZiBzdGRpby4gTG9va2luZyBhdCB0aGUgZGlmZiwgSSB3b25k ZXIgaWYgaXQgY2FuIGJlCisgICAgICAgIGZ1cnRoZXIgaW1wcm92ZWQsIGFzIGl0J3Mgbm90IHN1 cGVyIHJlYWRhYmxlLCBidXQgc3RkaW8gaXMgZW50aXJlbHkgdXNlbGVzcyBpbiB0aGlzIGNvbnRl eHQuCisKKyAgICAgICAgKiBCdWlsZFNsYXZlU3VwcG9ydC9idWlsZC53ZWJraXQub3JnLWNvbmZp Zy9wdWJsaWNfaHRtbC9kYXNoYm9hcmQvU2NyaXB0cy9CdWlsZGJvdC5qczoKKyAgICAgICAgKEJ1 aWxkYm90LnByb3RvdHlwZS5kYXNoYm9hcmRUZXN0UmVzdWx0c1VSTEZvckl0ZXJhdGlvbik6IEdl bmVyYXRlIGZvcm1hdHRlZCBkaWZmIGxpbmsuCisKKyAgICAgICAgKiBCdWlsZFNsYXZlU3VwcG9y dC9idWlsZC53ZWJraXQub3JnLWNvbmZpZy9wdWJsaWNfaHRtbC9kYXNoYm9hcmQvU2NyaXB0cy9C dWlsZGJvdEl0ZXJhdGlvbi5qczoKKyAgICAgICAgRml4ZWQgYSB0eXBvIGluIHN0ZXAgbmFtZSwg c28gdGhhdCBpdCdzIGFjdHVhbGx5IHRyZWF0ZWQgYXMgcHJvZHVjdGl2ZS4KKworICAgICAgICAq IEJ1aWxkU2xhdmVTdXBwb3J0L2J1aWxkLndlYmtpdC5vcmctY29uZmlnL3B1YmxpY19odG1sL2Rh c2hib2FyZC9TY3JpcHRzL0J1aWxkYm90VGVzdGVyUXVldWVWaWV3LmpzOgorICAgICAgICAoQnVp bGRib3RUZXN0ZXJRdWV1ZVZpZXcucHJvdG90eXBlLmFwcGVuZEJ1aWxkZXJRdWV1ZVN0YXR1cyk6 CisgICAgICAgIChCdWlsZGJvdFRlc3RlclF1ZXVlVmlldy5wcm90b3R5cGUudXBkYXRlKToKKyAg ICAgICAgKEJ1aWxkYm90VGVzdGVyUXVldWVWaWV3LnByb3RvdHlwZS5fcHJlc2VudFBvcG92ZXJG b3JHZW5lcmljVGVzdEZhaWx1cmVzKToKKyAgICAgICAgQWRkZWQgc3BlY2lhbCBoYW5kbGluZyBm b3IgZGFzaGJvYXJkIHRlc3QgcmVzdWx0cy4KKwogMjAxNy0wMy0yNyAgTHVjYXMgRm9yc2NobGVy ICA8bGZvcnNjaGxlckBhcHBsZS5jb20+CiAKICAgICAgICAgQW5vdGhlciB0ZXN0IHYyLgpJbmRl eDogVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvYnVpbGQud2Via2l0Lm9yZy1jb25maWcvcHVibGlj X2h0bWwvZGFzaGJvYXJkL1NjcmlwdHMvQnVpbGRib3QuanMKPT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMv QnVpbGRTbGF2ZVN1cHBvcnQvYnVpbGQud2Via2l0Lm9yZy1jb25maWcvcHVibGljX2h0bWwvZGFz aGJvYXJkL1NjcmlwdHMvQnVpbGRib3QuanMJKHJldmlzaW9uIDIxNDQ0NikKKysrIFRvb2xzL0J1 aWxkU2xhdmVTdXBwb3J0L2J1aWxkLndlYmtpdC5vcmctY29uZmlnL3B1YmxpY19odG1sL2Rhc2hi b2FyZC9TY3JpcHRzL0J1aWxkYm90LmpzCSh3b3JraW5nIGNvcHkpCkBAIC0xNzksNiArMTc5LDEx IEBAIEJ1aWxkYm90LnByb3RvdHlwZSA9IHsKICAgICAgICAgcmV0dXJuIHRoaXMubGF5b3V0VGVz dFJlc3VsdHNEaXJlY3RvcnlVUkxGb3JJdGVyYXRpb24oaXRlcmF0aW9uKSArICIvcmVzdWx0cy5o dG1sIjsKICAgICB9LAogCisgICAgZGFzaGJvYXJkVGVzdFJlc3VsdHNVUkxGb3JJdGVyYXRpb246 IGZ1bmN0aW9uKGl0ZXJhdGlvbikKKyAgICB7CisgICAgICAgIHJldHVybiB0aGlzLmxheW91dFRl c3RSZXN1bHRzRGlyZWN0b3J5VVJMRm9ySXRlcmF0aW9uKGl0ZXJhdGlvbikgKyAiL2Rhc2hib2Fy ZC1sYXlvdXQtdGVzdC1yZXN1bHRzL2luZGV4LXByZXR0eS1kaWZmLmh0bWwiOworICAgIH0sCisK ICAgICBsYXlvdXRUZXN0RnVsbFJlc3VsdHNVUkxGb3JJdGVyYXRpb246IGZ1bmN0aW9uKGl0ZXJh dGlvbikKICAgICB7CiAgICAgICAgIHJldHVybiB0aGlzLmxheW91dFRlc3RSZXN1bHRzRGlyZWN0 b3J5VVJMRm9ySXRlcmF0aW9uKGl0ZXJhdGlvbikgKyAiL2Z1bGxfcmVzdWx0cy5qc29uIjsKSW5k ZXg6IFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2J1aWxkLndlYmtpdC5vcmctY29uZmlnL3B1Ymxp Y19odG1sL2Rhc2hib2FyZC9TY3JpcHRzL0J1aWxkYm90SXRlcmF0aW9uLmpzCj09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0K LS0tIFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2J1aWxkLndlYmtpdC5vcmctY29uZmlnL3B1Ymxp Y19odG1sL2Rhc2hib2FyZC9TY3JpcHRzL0J1aWxkYm90SXRlcmF0aW9uLmpzCShyZXZpc2lvbiAy MTQ0NDYpCisrKyBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9idWlsZC53ZWJraXQub3JnLWNvbmZp Zy9wdWJsaWNfaHRtbC9kYXNoYm9hcmQvU2NyaXB0cy9CdWlsZGJvdEl0ZXJhdGlvbi5qcwkod29y a2luZyBjb3B5KQpAQCAtODEsNyArODEsNyBAQCBCdWlsZGJvdEl0ZXJhdGlvbi5UZXN0U3RlcHMg PSB7CiAgICAgImxheW91dC10ZXN0IjogImxheW91dCB0ZXN0IiwKICAgICAicGVyZi10ZXN0Ijog InBlcmZvcm1hbmNlIHRlc3QiLAogICAgICJydW4tYXBpLXRlc3RzIjogImFwaSB0ZXN0IiwKLSAg ICAicnVuLWRhc2hib2FyZC10ZXN0cyI6ICJkYXNoYm9hcmQgdGVzdCIsCisgICAgImRhc2hib2Fy ZC10ZXN0cyI6ICJkYXNoYm9hcmQgdGVzdCIsCiAgICAgIndlYmtpdC0zMmJpdC1qc2MtdGVzdCI6 ICJqYXZhc2NyaXB0IHRlc3QiLAogICAgICJ3ZWJraXQtanNjLWNsb29wLXRlc3QiOiAiamF2YXNj cmlwdCBjbG9vcCB0ZXN0IiwKICAgICAid2Via2l0cGVybC10ZXN0IjogIndlYmtpdHBlcmwgdGVz dCIsCkluZGV4OiBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9idWlsZC53ZWJraXQub3JnLWNvbmZp Zy9wdWJsaWNfaHRtbC9kYXNoYm9hcmQvU2NyaXB0cy9CdWlsZGJvdFRlc3RlclF1ZXVlVmlldy5q cwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09Ci0tLSBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9idWlsZC53ZWJraXQub3Jn LWNvbmZpZy9wdWJsaWNfaHRtbC9kYXNoYm9hcmQvU2NyaXB0cy9CdWlsZGJvdFRlc3RlclF1ZXVl Vmlldy5qcwkocmV2aXNpb24gMjE0NDQ2KQorKysgVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvYnVp bGQud2Via2l0Lm9yZy1jb25maWcvcHVibGljX2h0bWwvZGFzaGJvYXJkL1NjcmlwdHMvQnVpbGRi b3RUZXN0ZXJRdWV1ZVZpZXcuanMJKHdvcmtpbmcgY29weSkKQEAgLTEwOCw2ICsxMDgsOSBAQCBC dWlsZGJvdFRlc3RlclF1ZXVlVmlldy5wcm90b3R5cGUgPSB7CiAgICAgICAgICAgICAgICAgICAg IH0gZWxzZSBpZiAoLyg/PS4qdGVzdCkoPz0uKmpzYykvLnRlc3QoZmFpbGVkU3RlcC5uYW1lKSkg ewogICAgICAgICAgICAgICAgICAgICAgICAgdmFyIHN0YXR1cyA9IG5ldyBTdGF0dXNMaW5lVmll dyhtZXNzYWdlRWxlbWVudCwgU3RhdHVzTGluZVZpZXcuU3RhdHVzLkJhZCwgdGhpcy5fdGVzdFN0 ZXBGYWlsdXJlRGVzY3JpcHRpb24oZmFpbGVkU3RlcCksIGZhaWxlZFN0ZXAuZmFpbHVyZUNvdW50 LCBpdGVyYXRpb24ucXVldWUuYnVpbGRib3QuamF2YVNjcmlwdENvcmVUZXN0U3RkaW9VcmxGb3JJ dGVyYXRpb24oaXRlcmF0aW9uLCBmYWlsZWRTdGVwLm5hbWUpKTsKICAgICAgICAgICAgICAgICAg ICAgICAgIG5ldyBQb3BvdmVyVHJhY2tlcihzdGF0dXMuc3RhdHVzQnViYmxlRWxlbWVudCwgdGhp cy5fcHJlc2VudFBvcG92ZXJGb3JKYXZhU2NyaXB0Q29yZVRlc3RSZWdyZXNzaW9ucy5iaW5kKHRo aXMsIGZhaWxlZFN0ZXAubmFtZSksIGl0ZXJhdGlvbik7CisgICAgICAgICAgICAgICAgICAgIH0g ZWxzZSBpZiAoZmFpbGVkU3RlcC5uYW1lID09PSAiZGFzaGJvYXJkLXRlc3RzIikgeworICAgICAg ICAgICAgICAgICAgICAgICAgdmFyIHN0YXR1cyA9IG5ldyBTdGF0dXNMaW5lVmlldyhtZXNzYWdl RWxlbWVudCwgU3RhdHVzTGluZVZpZXcuU3RhdHVzLkJhZCwgdGhpcy5fdGVzdFN0ZXBGYWlsdXJl RGVzY3JpcHRpb24oZmFpbGVkU3RlcCksIHVuZGVmaW5lZCwgaXRlcmF0aW9uLnF1ZXVlLmJ1aWxk Ym90LmRhc2hib2FyZFRlc3RSZXN1bHRzVVJMRm9ySXRlcmF0aW9uKGl0ZXJhdGlvbikpOworICAg ICAgICAgICAgICAgICAgICAgICAgbmV3IFBvcG92ZXJUcmFja2VyKHN0YXR1cy5zdGF0dXNCdWJi bGVFbGVtZW50LCB0aGlzLl9wcmVzZW50UG9wb3ZlckZvckdlbmVyaWNUZXN0RmFpbHVyZXMuYmlu ZCh0aGlzKSwgaXRlcmF0aW9uKTsKICAgICAgICAgICAgICAgICAgICAgfSBlbHNlIHsKICAgICAg ICAgICAgICAgICAgICAgICAgIHZhciBzdGF0dXMgPSBuZXcgU3RhdHVzTGluZVZpZXcobWVzc2Fn ZUVsZW1lbnQsIFN0YXR1c0xpbmVWaWV3LlN0YXR1cy5CYWQsIHRoaXMuX3Rlc3RTdGVwRmFpbHVy ZURlc2NyaXB0aW9uKGZhaWxlZFN0ZXApLCBmYWlsZWRTdGVwLmZhaWx1cmVDb3VudCA/IGZhaWxl ZFN0ZXAuZmFpbHVyZUNvdW50IDogdW5kZWZpbmVkLCBmYWlsZWRTdGVwLlVSTCk7CiAgICAgICAg ICAgICAgICAgICAgICAgICBuZXcgUG9wb3ZlclRyYWNrZXIoc3RhdHVzLnN0YXR1c0J1YmJsZUVs ZW1lbnQsIHRoaXMuX3ByZXNlbnRQb3BvdmVyRm9yR2VuZXJpY1Rlc3RGYWlsdXJlcy5iaW5kKHRo aXMpLCBpdGVyYXRpb24pOwpAQCAtMjUyLDYgKzI1NSw4IEBAIEJ1aWxkYm90VGVzdGVyUXVldWVW aWV3LnByb3RvdHlwZSA9IHsKICAgICAgICAgaXRlcmF0aW9uLmZhaWxlZFRlc3RTdGVwcy5mb3JF YWNoKGZ1bmN0aW9uKGZhaWxlZFN0ZXApIHsKICAgICAgICAgICAgIGlmIChmYWlsZWRTdGVwLm5h bWUgPT09ICJsYXlvdXQtdGVzdCIpCiAgICAgICAgICAgICAgICAgYWRkUmVzdWx0S2luZCh0aGlz Ll90ZXN0U3RlcEZhaWx1cmVEZXNjcmlwdGlvbldpdGhDb3VudChmYWlsZWRTdGVwKSwgaXRlcmF0 aW9uLnF1ZXVlLmJ1aWxkYm90LmxheW91dFRlc3RSZXN1bHRzVVJMRm9ySXRlcmF0aW9uKGl0ZXJh dGlvbikpOworICAgICAgICAgICAgaWYgKGZhaWxlZFN0ZXAubmFtZSA9PT0gImRhc2hib2FyZC10 ZXN0cyIpCisgICAgICAgICAgICAgICAgYWRkUmVzdWx0S2luZCh0aGlzLl90ZXN0U3RlcEZhaWx1 cmVEZXNjcmlwdGlvbihmYWlsZWRTdGVwKSwgaXRlcmF0aW9uLnF1ZXVlLmJ1aWxkYm90LmRhc2hi b2FyZFRlc3RSZXN1bHRzVVJMRm9ySXRlcmF0aW9uKGl0ZXJhdGlvbikpOwogICAgICAgICAgICAg ZWxzZQogICAgICAgICAgICAgICAgIGFkZFJlc3VsdEtpbmQodGhpcy5fdGVzdFN0ZXBGYWlsdXJl RGVzY3JpcHRpb25XaXRoQ291bnQoZmFpbGVkU3RlcCksIGZhaWxlZFN0ZXAuVVJMKTsKICAgICAg ICAgfSwgdGhpcyk7Cg==