➜ build.webkit.org-config python steps_unittest.py import sets as _sets ..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................FFFFF............FF.................F................................... ====================================================================== FAIL: test_builder Apple Win 10 Debug (Build) (__main__.BuildStepsTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "steps_unittest.py", line 598, in doTest self.assertTrue(builder['name'] in expected_build_steps, "Missing expected result for builder: %s\n Actual result is %s" % (builder['name'], buildSteps)) AssertionError: Missing expected result for builder: Apple Win 10 Debug (Build) Actual result is ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile', 'compile-webkit', 'archive-built-product', 'upload', 'transfer-to-s3', 'trigger'] ====================================================================== FAIL: test_builder Apple Win 10 Debug (Tests) (__main__.BuildStepsTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "steps_unittest.py", line 598, in doTest self.assertTrue(builder['name'] in expected_build_steps, "Missing expected result for builder: %s\n Actual result is %s" % (builder['name'], buildSteps)) AssertionError: Missing expected result for builder: Apple Win 10 Debug (Tests) Actual result is ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile', 'download-built-product', 'extract-built-product', 'jscore-test', 'layout-test', 'run-api-tests', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'archive-test-results', 'upload', 'MasterShellCommand'] ====================================================================== FAIL: test_builder Apple Win 10 Release (Build) (__main__.BuildStepsTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "steps_unittest.py", line 598, in doTest self.assertTrue(builder['name'] in expected_build_steps, "Missing expected result for builder: %s\n Actual result is %s" % (builder['name'], buildSteps)) AssertionError: Missing expected result for builder: Apple Win 10 Release (Build) Actual result is ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile', 'compile-webkit', 'archive-built-product', 'upload', 'transfer-to-s3', 'trigger'] ====================================================================== FAIL: test_builder Apple Win 10 Release (Tests) (__main__.BuildStepsTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "steps_unittest.py", line 598, in doTest self.assertTrue(builder['name'] in expected_build_steps, "Missing expected result for builder: %s\n Actual result is %s" % (builder['name'], buildSteps)) AssertionError: Missing expected result for builder: Apple Win 10 Release (Tests) Actual result is ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile', 'download-built-product', 'extract-built-product', 'jscore-test', 'layout-test', 'run-api-tests', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'archive-test-results', 'upload', 'MasterShellCommand'] ====================================================================== FAIL: test_builder Apple Win 7 Release (Build) (__main__.BuildStepsTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "steps_unittest.py", line 598, in doTest self.assertTrue(builder['name'] in expected_build_steps, "Missing expected result for builder: %s\n Actual result is %s" % (builder['name'], buildSteps)) AssertionError: Missing expected result for builder: Apple Win 7 Release (Build) Actual result is ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile', 'compile-webkit', 'archive-built-product', 'upload', 'transfer-to-s3', 'trigger'] ====================================================================== FAIL: test_builder GTK Linux 64-bit Release Debian Stable (Build) (__main__.BuildStepsTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "steps_unittest.py", line 599, in doTest self.assertListEqual(expected_build_steps[builder['name']], buildSteps) AssertionError: Lists differ: ['configure build', 'svn', 'ki... != ['configure build', 'svn', 'ki... First differing element 5: compile-webkit jhbuild Second list contains 1 additional elements. First extra element 6: compile-webkit ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', + 'jhbuild', 'compile-webkit'] ====================================================================== FAIL: test_builder GTK Linux 64-bit Release Ubuntu LTS (Build) (__main__.BuildStepsTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "steps_unittest.py", line 599, in doTest self.assertListEqual(expected_build_steps[builder['name']], buildSteps) AssertionError: Lists differ: ['configure build', 'svn', 'ki... != ['configure build', 'svn', 'ki... First differing element 5: compile-webkit jhbuild Second list contains 1 additional elements. First extra element 6: compile-webkit ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', + 'jhbuild', 'compile-webkit'] ====================================================================== FAIL: test_unnecessary_expected_results (__main__.BuildStepsTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "steps_unittest.py", line 609, in test_unnecessary_expected_results self.assertTrue(builder in builders, "Builder %s doesn't exist, but has unnecessary expected results" % builder) AssertionError: Builder Apple Win Release (Build) doesn't exist, but has unnecessary expected results ---------------------------------------------------------------------- Ran 902 tests in 0.028s FAILED (failures=8)
Created attachment 353936 [details] Patch
Comment on attachment 353936 [details] Patch Why is there a change to GTK steps? Did that change earlier?
(In reply to Alexey Proskuryakov from comment #2) > Comment on attachment 353936 [details] > Patch > > Why is there a change to GTK steps? Did that change earlier? It appears to be due to: [GTK] Disable MSE build support on Debian and Ubuntu LTS bots https://bugs.webkit.org/show_bug.cgi?id=190581
The GTK/WPE change is wrong and unexpected. We were just trying to disable a build flag, not introduce a new build step. Are we sure this change was related to r237113? Anyway, I'm impressed that the test caught this problem. CC Adrian. Adrian, the problem is the Ubuntu LTS bots have unexpectedly started running a JHBuild environment. They're supposed to be using system dependencies only, not JHBuild.
(In reply to Michael Catanzaro from comment #4) > The GTK/WPE change is wrong and unexpected. We were just trying to disable a > build flag, not introduce a new build step. Are we sure this change was > related to r237113? Anyway, I'm impressed that the test caught this problem. I came to this conclusion using git bisect. Checking build history, the step appearance lines up with when I restarted buildbot on October 15th (https://bugs.webkit.org/show_bug.cgi?id=190581#c7). I'll just land the Win10/Win7 portion of my change.
Reported bug #191318 for the GTK problem.
I landed the Win10/Win7 test fixes in https://trac.webkit.org/changeset/237991/webkit
<rdar://problem/45917298>