Bug 40891 – Python Tests Fail after r61508

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Joseph Pecoraro]

http://trac.webkit.org/changeset/61508 causes a test failure:

        # This test should probably be updated if the default regexp list changes
        self.assertEquals(buildbot.core_builder_names_regexps, name_regexps)

The test's name_regexps needs to be updated.

[Joseph Pecoraro]

Created attachment 59211 [details]
[PATCH] Fix Tests

Anyone know why this assert is there instead of just using the list directly?

[Joseph Pecoraro]

Its likely that the test should also include new "Chromium" builders in example_builders. I'll see if I can find out the names of these and add them.

[Joseph Pecoraro]

Created attachment 59213 [details]
[PATCH] Fix Test and Update Example Bots List

I couldn't find a simple list of all the bots on build.webkit.org, so
I extracted the names of bots from:
http://build.webkit.org/buildslaves

I ran them through `sort | uniq` and update the example_builders
list. I then sorted the expected results. This does include a few
new Chromium bots which would have made Will's change necessary.

[Joseph Pecoraro]

Created attachment 59214 [details]
[DIFF] The Difference in the example_bots list

Here is the diff of the old list (sorted) an the new list (sorted).
It looks like just the Chromium bots were added.

[Csaba Osztrogonác]

I think you shouldn't modify the order of 
the bots, because same order in python file 
as on the buildbot master is more readable.

I prefer a patch like this:

diff --git a/WebKitTools/Scripts/webkitpy/common/net/buildbot_unittest.py b/WebKitTools/Scripts/webkitpy/common/net/buildbot_uni
index 5384321..2742040 100644
--- a/WebKitTools/Scripts/webkitpy/common/net/buildbot_unittest.py
+++ b/WebKitTools/Scripts/webkitpy/common/net/buildbot_unittest.py
@@ -275,6 +275,9 @@ class BuildBotTest(unittest.TestCase):
             {'name': u'Chromium Linux Release', },
             {'name': u'Chromium Mac Release', },
             {'name': u'Chromium Win Release', },
+            {'name': u'Chromium Linux Release (Tests)', },
+            {'name': u'Chromium Mac Release (Tests)', },
+            {'name': u'Chromium Win Release (Tests)', },
             {'name': u'New run-webkit-tests', },
         ]
         name_regexps = [
@@ -285,7 +288,7 @@ class BuildBotTest(unittest.TestCase):
             "Windows.*Build",
             "GTK",
             "Qt",
-            "Chromium",
+            "Chromium.*Release$",
         ]
         expected_builders = [
             {'name': u'Tiger Intel Release', },

[Joseph Pecoraro]

Created attachment 59216 [details]
[PATCH] Fix Test + Order of Bots on Full Waterfall

Took Ossy's suggestion.

[Dimitri Glazkov (Google)]

Comment on attachment 59216 [details]
[PATCH] Fix Test + Order of Bots on Full Waterfall

r=me, except:

WebKitTools/Scripts/webkitpy/common/net/buildbot_unittest.py:291
 +              "Chromium.*Release$",
I already landed that :)

[Joseph Pecoraro]

> WebKitTools/Scripts/webkitpy/common/net/buildbot_unittest.py:291
>  +              "Chromium.*Release$",
> I already landed that :)

I see, in:
http://trac.webkit.org/changeset/61512

Committed r61513
	M	WebKitTools/ChangeLog
	M	WebKitTools/Scripts/webkitpy/common/net/buildbot_unittest.py
r61513 = c3ff3c574713fdc4ffdb4220115e06adfc5599c3 (refs/remotes/trunk)
http://trac.webkit.org/changeset/61513(In reply to comment #7)

Thanks!

[William Siegrist]

Wouldn't a better fix be to only have the list(s) in one location? If you don't want the unit tests to require the buildbot master config, then you can make the buildbot config read the builder data from webkitpy (since the master has access to it already). 

In any case, sorry about the breakage.

[Adam Barth]

Yeah, I think the current system is kind of redundant, but testing usually introduces redundancies into the system.  @eric, what do you think?

[Eric Seidel (no email)]

We've had multiple people trip on these.  But they're also very useful for testing the regexps and making sure the core builder list is actually what you plan it to be. :)  So donno.