countFailures of RunLLINTCLoopTests and Run32bitJSCTests in master.cfg don't work at all, they don't report the number of failing tests: http://build.webkit.org/builders/Apple%20Mavericks%2032-bit%20JSC%20%28BuildAndTest%29/builds/3703 It's hard to determine which patch caused how many regressions if we can't see the exact number on the waterfall. The buggy countFailures introduced in: https://trac.webkit.org/changeset/166608 https://trac.webkit.org/changeset/161461 Why can't we reuse the RunJavaScriptCoreTests class which works fine and its commandComplete covered by unittests too?
I think only the 32 bit countFailures is buggy, because this bot runs the jsc-stress tests, not the old Mozilla tests, and their output are different. But unifying the output parsing of run-javascriptcore-tests would be great.
Run32bitJSCTests buildstep runs the JSC stress tests, so its regexp should accept "failure" and "failures", not "regressions". RunLLINTCLoopTests buildstep runs Mozilla tests, so its regexp should accept "regression" and "regressions" too. The command of RunJavaScriptCoreTests, RunLLINTCLoopTests and Run32bitJSCTests are a little bit different, so unifying them isn't trivial. So let's fix the regexps quickly and try to unify them later.
Created attachment 236923 [details] Proposed patch Quick fix in regexps at RunLLINTCLoopTests and Run32bitJSCTests class.
Attachment 236923 [details] did not pass style-queue: ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:148: [RunLLINTCLoopTestsTest.assertResults] Undefined variable 'RunLLINTCLoopTests' [pylint/E0602] [5] ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:157: [RunLLINTCLoopTestsTest.test_failures] Undefined variable 'FAILURE' [pylint/E0602] [5] ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:160: [RunLLINTCLoopTestsTest.test_failure] Undefined variable 'FAILURE' [pylint/E0602] [5] ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:163: [RunLLINTCLoopTestsTest.test_no_failure] Undefined variable 'SUCCESS' [pylint/E0602] [5] ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:169: [Run32bitJSCTestsTest.assertResults] Undefined variable 'Run32bitJSCTests' [pylint/E0602] [5] ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:178: [Run32bitJSCTestsTest.test_failures] Undefined variable 'FAILURE' [pylint/E0602] [5] ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:181: [Run32bitJSCTestsTest.test_failure] Undefined variable 'FAILURE' [pylint/E0602] [5] ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:184: [Run32bitJSCTestsTest.test_no_failure] Undefined variable 'SUCCESS' [pylint/E0602] [5] Total errors found: 8 in 3 files If any of these errors are false positives, please file a bug against check-webkit-style.
Comment on attachment 236923 [details] Proposed patch LGTM, just a nit: Can we use only single or double quotes?
Created attachment 236924 [details] Proposed patch Change all double quote to singe quote.
Attachment 236924 [details] did not pass style-queue: ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:148: [RunLLINTCLoopTestsTest.assertResults] Undefined variable 'RunLLINTCLoopTests' [pylint/E0602] [5] ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:157: [RunLLINTCLoopTestsTest.test_failures] Undefined variable 'FAILURE' [pylint/E0602] [5] ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:160: [RunLLINTCLoopTestsTest.test_failure] Undefined variable 'FAILURE' [pylint/E0602] [5] ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:163: [RunLLINTCLoopTestsTest.test_no_failure] Undefined variable 'SUCCESS' [pylint/E0602] [5] ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:169: [Run32bitJSCTestsTest.assertResults] Undefined variable 'Run32bitJSCTests' [pylint/E0602] [5] ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:178: [Run32bitJSCTestsTest.test_failures] Undefined variable 'FAILURE' [pylint/E0602] [5] ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:181: [Run32bitJSCTestsTest.test_failure] Undefined variable 'FAILURE' [pylint/E0602] [5] ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:184: [Run32bitJSCTestsTest.test_no_failure] Undefined variable 'SUCCESS' [pylint/E0602] [5] Total errors found: 8 in 3 files If any of these errors are false positives, please file a bug against check-webkit-style.
Comment on attachment 236924 [details] Proposed patch Clearing flags on attachment: 236924 Committed r172829: <http://trac.webkit.org/changeset/172829>
All reviewed patches have been landed. Closing bug.