Bug 65794 - Wire up updating expectations in garden-o-matic.
Summary: Wire up updating expectations in garden-o-matic.
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Dimitri Glazkov (Google)
URL:
Keywords:
Depends on:
Blocks: 64183
  Show dependency treegraph
 
Reported: 2011-08-05 15:36 PDT by Dimitri Glazkov (Google)
Modified: 2011-08-08 12:59 PDT (History)
1 user (show)

See Also:


Attachments
Patch (8.27 KB, patch)
2011-08-05 15:40 PDT, Dimitri Glazkov (Google)
no flags Details | Formatted Diff | Diff
Better. (14.17 KB, patch)
2011-08-08 12:26 PDT, Dimitri Glazkov (Google)
abarth: review+
abarth: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Dimitri Glazkov (Google) 2011-08-05 15:36:49 PDT
Wire up updating expectations in garden-o-matic.
Comment 1 Dimitri Glazkov (Google) 2011-08-05 15:40:00 PDT
Created attachment 103123 [details]
Patch
Comment 2 Adam Barth 2011-08-05 15:50:54 PDT
Comment on attachment 103123 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=103123&action=review

> Tools/Scripts/webkitpy/layout_tests/port/factory.py:40
> +    DEBUG_CONFIGURATION_REGEX = r"[d|D](ebu|b)g"

What's the point of having this be a class-level static?  I'd just keep it as a private variable to __init__

> Tools/Scripts/webkitpy/tool/servers/gardeningserver.py:35
> +# FIXME: This needs to be properly implemented.
> +class GardeningBugManager(object):

OMG, that's the worst name ever.  Does this class need to inherit from some sort of interface object?  Usually, we'd just have GardeningExpectationsController inherit from that interface and implement these methods.  Haven't a separate class doesn't seem to buy us that much.

> Tools/Scripts/webkitpy/tool/servers/gardeningserver.py:43
> +class GardeningExpectationsController(object):

This class does not appear to be tested.  Also, I'd call it ExpectationsUpdater.  :)

> Tools/Scripts/webkitpy/tool/servers/gardeningserver.py:44
> +    def __init__(self, filesystem):

This should probably take a whole tool b/c it's going to need tool.bugs at some point too.  If we were lower down in the dependency graph, we'd want to break out the various parts of the tool, but up here in the "tool" package, we usually just pass around the tool itself.

> Tools/Scripts/webkitpy/tool/servers/gardeningserver.py:51
> +        self.port = factory.get("chromium-win-win7")
> +        self.bug_manager = GardeningBugManager()
> +        self.converter = TestConfigurationConverter(self.port.all_test_configurations(), self.port.configuration_specifier_macros())
> +        self.parser = TestExpectationParser(self.port, [], allow_rebaseline_modifier=False)

If these are all private, then should start with an _ character.

> Tools/Scripts/webkitpy/tool/servers/gardeningserver.py:63
> +    def _builder_to_test_config(self, builder_name):
> +        return factory.get_from_builder_name(builder_name).test_configuration()

@memoize.  You're going to be calling this sucker a lot because you call it from the for failure_info in failure_info_list loop.

> Tools/Scripts/webkitpy/tool/servers/gardeningserver.py:70
> +        self.expectations_controller = GardeningExpectationsController(self.tool.filesystem)

Rather than keeping this on the server, I'd use a memoized function on GardeningHTTPRequestHandler that creates the object.
Comment 3 Adam Barth 2011-08-05 16:16:46 PDT
Comment on attachment 103123 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=103123&action=review

> Tools/Scripts/webkitpy/tool/servers/gardeningserver.py:59
> +            editor.update_expectations(failure_info.testName, self._builder_to_test_config(failure_info.builderName), set(failure_info.failureTypeList))

Doesn't this need to be failure_info['testName'] ?  This isn't JavaScript.  There are rules!
Comment 4 Dimitri Glazkov (Google) 2011-08-08 10:41:15 PDT
Comment on attachment 103123 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=103123&action=review

>> Tools/Scripts/webkitpy/layout_tests/port/factory.py:40
>> +    DEBUG_CONFIGURATION_REGEX = r"[d|D](ebu|b)g"
> 
> What's the point of having this be a class-level static?  I'd just keep it as a private variable to __init__

Fixed.

>> Tools/Scripts/webkitpy/tool/servers/gardeningserver.py:35
>> +class GardeningBugManager(object):
> 
> OMG, that's the worst name ever.  Does this class need to inherit from some sort of interface object?  Usually, we'd just have GardeningExpectationsController inherit from that interface and implement these methods.  Haven't a separate class doesn't seem to buy us that much.

Moved to GardeningExpectationsUpdater.

>> Tools/Scripts/webkitpy/tool/servers/gardeningserver.py:43
>> +class GardeningExpectationsController(object):
> 
> This class does not appear to be tested.  Also, I'd call it ExpectationsUpdater.  :)

Renamed, adding unit tests now.

>> Tools/Scripts/webkitpy/tool/servers/gardeningserver.py:44
>> +    def __init__(self, filesystem):
> 
> This should probably take a whole tool b/c it's going to need tool.bugs at some point too.  If we were lower down in the dependency graph, we'd want to break out the various parts of the tool, but up here in the "tool" package, we usually just pass around the tool itself.

Fixed.

>> Tools/Scripts/webkitpy/tool/servers/gardeningserver.py:51
>> +        self.parser = TestExpectationParser(self.port, [], allow_rebaseline_modifier=False)
> 
> If these are all private, then should start with an _ character.

Fixed.

>> Tools/Scripts/webkitpy/tool/servers/gardeningserver.py:59
>> +            editor.update_expectations(failure_info.testName, self._builder_to_test_config(failure_info.builderName), set(failure_info.failureTypeList))
> 
> Doesn't this need to be failure_info['testName'] ?  This isn't JavaScript.  There are rules!

Rules are for the weak.

>> Tools/Scripts/webkitpy/tool/servers/gardeningserver.py:63
>> +        return factory.get_from_builder_name(builder_name).test_configuration()
> 
> @memoize.  You're going to be calling this sucker a lot because you call it from the for failure_info in failure_info_list loop.

Fixed.

>> Tools/Scripts/webkitpy/tool/servers/gardeningserver.py:70
>> +        self.expectations_controller = GardeningExpectationsController(self.tool.filesystem)
> 
> Rather than keeping this on the server, I'd use a memoized function on GardeningHTTPRequestHandler that creates the object.

Ooh, pretty. Done.
Comment 5 Dimitri Glazkov (Google) 2011-08-08 12:26:32 PDT
Created attachment 103276 [details]
Better.
Comment 6 Adam Barth 2011-08-08 12:33:56 PDT
Comment on attachment 103276 [details]
Better.

View in context: https://bugs.webkit.org/attachment.cgi?id=103276&action=review

> Tools/Scripts/webkitpy/tool/servers/gardeningserver.py:59
> +            expectation_set = set([expectation for expectation in map(TestExpectations.expectation_from_string, failure_info['failureTypeList']) if expectation])

This looks like just map + filter.

> Tools/Scripts/webkitpy/tool/servers/gardeningserver.py:63
> +            editor.update_expectation(failure_info['testName'], set([self._builder_to_test_config(failure_info['builderName'])]), expectation_set)

failure_info['testName'] -> test_name

> Tools/Scripts/webkitpy/tool/servers/gardeningserver_unittest.py:55
> +    @classmethod
> +    def path_to_test_expectations_file(cls):
> +        if not cls._path_to_test_expectations_file:
> +            cls._path_to_test_expectations_file = cls.create().path_to_test_expectations_file()
> +        return cls._path_to_test_expectations_file

This looks like a manual memoize.  Does this really need to be cached?  How much does it slow down the test not to be cached?
Comment 7 Dimitri Glazkov (Google) 2011-08-08 12:59:48 PDT
Committed r92624: <http://trac.webkit.org/changeset/92624>