Summary: | [buildbot] Add regression test to check which builder runs which tests | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | WebKit | Reporter: | Csaba Osztrogonác <ossy> | ||||||
Component: | Tools / Tests | Assignee: | Csaba Osztrogonác <ossy> | ||||||
Status: | RESOLVED FIXED | ||||||||
Severity: | Normal | CC: | ap, commit-queue, dburkart, lforschler, matthew_hanson, mmirman, ossy, rniwa | ||||||
Priority: | P2 | ||||||||
Version: | 528+ (Nightly build) | ||||||||
Hardware: | Unspecified | ||||||||
OS: | Unspecified | ||||||||
Attachments: |
|
Description
Csaba Osztrogonác
2015-03-02 07:46:44 PST
Created attachment 247668 [details]
Patch
Attachment 247668 [details] did not pass style-queue:
ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:368: whitespace before ':' [pep8/E203] [5]
ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:415: [BuildStepsTest.generateTests] Undefined variable 'c' [pylint/E0602] [5]
ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:432: [BuildStepsTest.test_unnecessary_expected_results] Undefined variable 'c' [pylint/E0602] [5]
Total errors found: 3 in 2 files
If any of these errors are false positives, please file a bug against check-webkit-style.
Comment on attachment 247668 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=247668&action=review Seems like a good idea. What runs these tests, does buildbot's checkconfig do that? > Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:406 > + "GTK Linux ARM Release" : ["configure build", "svn", "kill old processes", "delete WebKitBuild directory", "delete stale build files", "jhbuild", "compile-webkit", "jscore-test", "webkitpy-test", "webkitperl-test", "bindings-generation-tests", "API tests", "WebKit GObject DOM bindings API break tests"], "WebKit GObject DOM bindings API break tests"? The dashboard doesn't know about this one. Comment on attachment 247668 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=247668&action=review >> Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:406 >> + "GTK Linux ARM Release" : ["configure build", "svn", "kill old processes", "delete WebKitBuild directory", "delete stale build files", "jhbuild", "compile-webkit", "jscore-test", "webkitpy-test", "webkitperl-test", "bindings-generation-tests", "API tests", "WebKit GObject DOM bindings API break tests"], > > "WebKit GObject DOM bindings API break tests"? The dashboard doesn't know about this one. I don't know what is this test for, but GTK bots run it with RunGtkWebKitGObjectDOMBindingsAPIBreakTests(): http://trac.webkit.org/browser/trunk/Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg?rev=180288#L600 (In reply to comment #4) > Comment on attachment 247668 [details] > Patch > > Seems like a good idea. What runs these tests, does buildbot's checkconfig > do that? Nor checkconfig runs it, nor Tools/Scripts/test-webkitpy, but it can be run by hand after a little patching. The actually used buildmaster is too old for actual twisted on Mavericks and Ubuntu 14.04. Unfortunately you have to fix the autoinstalled buildbot locally to be able to run the unittests. Tools/Scripts/webkitpy/thirdparty/autoinstalled/buildbot/ * status/web/changes.py: * status/web/logs.py: * status/web/slaves.py: -from twisted.web.error import NoResource +from twisted.web.resource import NoResource I think that it goes without saying that we need to run these tests automatically; there is no way anyone would remember to do it manually with hacks. (In reply to comment #7) > I think that it goes without saying that we need to run these tests > automatically; there is no way anyone would remember to do it manually with > hacks. I agree, will investigate how can we do it easily. I think we should try to upgrade the buildmaster. Now we still use 0.8.6p1 which was released was released on March 25, 2012 - nearly 3 years ago. I'll check which newer buildmaster release can we use on Mavericks and Ubuntu 14.04 too without too much adjust needed in master.cfg. Comment on attachment 247668 [details]
Patch
cq- now, I'm going to tweak a little bit to be able update resulults easily.
Created attachment 247764 [details] Patch Additionally dumping the actual result if expected result is missing (new bot added). And updated results after bug142094 (not landed yet). Attachment 247764 [details] did not pass style-queue:
ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:368: whitespace before ':' [pep8/E203] [5]
ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:417: [BuildStepsTest.generateTests] Undefined variable 'c' [pylint/E0602] [5]
ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:434: [BuildStepsTest.test_unnecessary_expected_results] Undefined variable 'c' [pylint/E0602] [5]
Total errors found: 3 in 2 files
If any of these errors are false positives, please file a bug against check-webkit-style.
(In reply to comment #7) > I think that it goes without saying that we need to run these tests > automatically; there is no way anyone would remember to do it manually with > hacks. I filed a new bug report to make it runnable out-of-the-box: bug142219 Comment on attachment 247764 [details] Patch Clearing flags on attachment: 247764 Committed r181070: <http://trac.webkit.org/changeset/181070> All reviewed patches have been landed. Closing bug. |