Bug 73531 – [meta] Make prepare-ChangeLog a Perl module

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Kentaro Hara]

In order to write unit tests for prepare-ChangeLog, the unit tests need to load prepare-ChangeLog as a Perl module.

[Kentaro Hara]

Created attachment 117371 [details]
Patch

[David Kilzer (:ddkilzer)]

Comment on attachment 117371 [details]
Patch

This patch tries to do too much at once.  If we're going to refactor prepare-ChangeLog, we should do it in smaller steps (with unit tests written for each smaller patch).

It would also be acceptable to change prepare-ChangeLog just enough to make it possible to load the file as a module for unit testing (like run-leaks).  The way you do this is (likely) to refactor away the use of the problematic global variables.

[Kentaro Hara]

(In reply to comment #2)
> (From update of attachment 117371 [details])
> This patch tries to do too much at once.  If we're going to refactor prepare-ChangeLog, we should do it in smaller steps (with unit tests written for each smaller patch).
> 
> It would also be acceptable to change prepare-ChangeLog just enough to make it possible to load the file as a module for unit testing (like run-leaks).  The way you do this is (likely) to refactor away the use of the problematic global variables.

I guess that this is a minimal change for making prepare-ChangeLog just work as a Perl module. On the other hand, I do agree that this patch is doing too much at once. Then, I'd like to split this patch into the following sub-patches:

[1] Moves top level code (outside any subroutines) into prepareChangeLog().
[2] Moves all non-constant global variables' initialization into InitializeGlobalData().
[3] Adds ChangeLog::new().
[4] Adds "package ChangeLog;" at the beginning and "1;" at the end. Rename resolve-ChangeLogs to ChangeLog.pm. Adds a new prepare-ChangeLog, which instantiates a ChangeLog object using new() and calls ChangeLog::prepareChangeLog().

[David Kilzer (:ddkilzer)]

(In reply to comment #3)
> (In reply to comment #2)
> > (From update of attachment 117371 [details] [details])
> > This patch tries to do too much at once.  If we're going to refactor prepare-ChangeLog, we should do it in smaller steps (with unit tests written for each smaller patch).
> > 
> > It would also be acceptable to change prepare-ChangeLog just enough to make it possible to load the file as a module for unit testing (like run-leaks).  The way you do this is (likely) to refactor away the use of the problematic global variables.
> 
> I guess that this is a minimal change for making prepare-ChangeLog just work as a Perl module. On the other hand, I do agree that this patch is doing too much at once. Then, I'd like to split this patch into the following sub-patches:
> 
> [1] Moves top level code (outside any subroutines) into prepareChangeLog().

The patches so far that do this look good.

> [2] Moves all non-constant global variables' initialization into InitializeGlobalData().
> [3] Adds ChangeLog::new().
> [4] Adds "package ChangeLog;" at the beginning and "1;" at the end. Rename resolve-ChangeLogs to ChangeLog.pm. Adds a new prepare-ChangeLog, which instantiates a ChangeLog object using new() and calls ChangeLog::prepareChangeLog().

I'm not sure Steps 2-4 are necessary.  Remember, we're able to test the run-leaks script without making it into a Perl module.  At some point, the changes from Step 1 should be enough to use the same technique as we're using with the run-leaks script.  At most, you may need a Step 1.1 and 1.2:

[1.1] Move remaining top-level code into a main() method.
[1.2] Remove most (all?) global variables from the script by moving them into main().

After those steps, please try using the run-leaks method of testing the Perl script again.

[Kentaro Hara]

(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #2)
> I'm not sure Steps 2-4 are necessary.  Remember, we're able to test the run-leaks script without making it into a Perl module.  At some point, the changes from Step 1 should be enough to use the same technique as we're using with the run-leaks script.  At most, you may need a Step 1.1 and 1.2:
> 
> [1.1] Move remaining top-level code into a main() method.
> [1.2] Remove most (all?) global variables from the script by moving them into main().
> 
> After those steps, please try using the run-leaks method of testing the Perl script again.

ddkilzer: Thanks. I am not opposed to the run-leaks approach but I still feel that making prepare-ChangeLog.pm a Perl module might be better.

- In the run-leaks approach, we need to prepare RunLeaks.pm for unit-testing. If run-leaks were a Perl module, RunLeaks.pm is not necessary.
- In case of prepare-ChangeLog, since there are already many global variables, removing all global variables will lead to more arguments on methods, e.g. diffCommand(.., $isSVN, $isGit, $gitCommit, $mergeBase). Rather than increasing arguments, using an object might be more readable, like this:

sub diffCommand(...)
{
    my $self = shift;
    if ($self->{isSVN}) {
        ...;
    } elsif ($self->{isGit}) {
        ...;
    }
}

WDYT?

[David Kilzer (:ddkilzer)]

(In reply to comment #5)
> ddkilzer: Thanks. I am not opposed to the run-leaks approach but I still feel that making prepare-ChangeLog.pm a Perl module might be better.

I agree that having a nice object-oriented code base for prepare-ChangeLog would be preferable to a monolithic script with lots of functions, but moving all the functions from prepare-ChangeLog into prepareChangeLog.pm does not provide a good OO model for the code.  The fact that the ChangeLog::new() method had 15 arguments is a code smell that moving all the code into a module wasn't the correct direction to take.

For example, a ChangeLog.pm module might represent a ChangeLog file, and most of the things you can do with it.  A new ChangeLog object would take a path to an existing ChangeLog, and then there would be methods to modify it, and write it back out again.

The logic for parsing existing file types might be a separate set of modules, or even a module (class) hierarchy since most would probably share a lot of code.

> - In the run-leaks approach, we need to prepare RunLeaks.pm for unit-testing. If run-leaks were a Perl module, RunLeaks.pm is not necessary.

While this is a bit ugly, it's small, it won't change much, and it let me accomplish (a) writing tests for existing functionality to prevent further regressions and (b) writing tests for the bug fix I wanted to make.

> - In case of prepare-ChangeLog, since there are already many global variables, removing all global variables will lead to more arguments on methods,

True.  The same thing happen when most of prepare-ChangeLog was moved into a module with a new() method that had 15 arguments in the initial patch on this bug.

> e.g. diffCommand(.., $isSVN, $isGit, $gitCommit, $mergeBase). Rather than increasing arguments, using an object might be more readable, like this:
> 
> sub diffCommand(...)
> {
>     my $self = shift;
>     if ($self->{isSVN}) {
>         ...;
>     } elsif ($self->{isGit}) {
>         ...;
>     }
> }
> 
> WDYT?

I was hoping that a small-to-moderate amount of refactoring of prepare-ChangeLog (moving code into methods and removing global variables) would make it testable like run-leaks, then you could fix the original bug.

I agree that moving the code into modules would help to clean it up, but I think some more design needs to be done rather than just moving all of the code into a *.pm file.

Other possible directions could be:

- Refactoring prepare-ChangeLog into modules.  (This reduces the hackability of the script, which was one of the original goals when it was written by Darin/Maciej, but would make it easier to maintain.)

- Extract just the parsing code into a module (or set of modules) to make it testable, then fix the bug.

- Rewrite it in Python since that appears to be the overall direction of the WebKit project (tools written in Python).  This would be a much larger effort, though.

I'm very happy that you're refactoring this code, but I also don't want it to become a full-time job.

[Kentaro Hara]

(In reply to comment #6)
> (In reply to comment #5)
> I was hoping that a small-to-moderate amount of refactoring of prepare-ChangeLog (moving code into methods and removing global variables) would make it testable like run-leaks, then you could fix the original bug.
> 
> I agree that moving the code into modules would help to clean it up, but I think some more design needs to be done rather than just moving all of the code into a *.pm file.

ddkilzer: Makes great sense. Thanks. I'll take the run-leaks approach.

> I'm very happy that you're refactoring this code, but I also don't want it to become a full-time job.

Yeah, we should avoid "over-engineering". The original objective is just to fix a bug of prepare-ChangeLog when it parses here-document in Perl, which makes my full time job (i.e. implementing IDL bindings in around CodeGenerator*.pm) frustrating:-) But anyway I am happy to take this opportunity to make prepare-ChangeLog sane using my free time.

[Kentaro Hara]

Let me close this meta bug, since we've added unittests for prepare-ChangeLog and solved an original issue.

[Hajime Morrita]

Comment on attachment 117371 [details]
Patch

Making this patch obsolete then.