Bug 36591 – test-webkitpy: should automatically detect all unit test files in webkitpy

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Chris Jerdonek]

This should make it unnecessary to have and maintain all of the unittests.py files.

test-webkitpy should be able to traverse webkitpy/, look for all files of the form *_unittest.py, and then import * from those files and call unittest.main().

[Chris Jerdonek]

See this page:

http://technogeek.org/python-module.html

Since we need to execute statements of the form "from ..._unittest import *", we may want to use the pattern below as suggested in the link above:

import_string = "from string import ascii_letters"
exec import_string

[Eric Seidel (no email)]

No need for "exec".  We can just import the module and then use defs() to grab all the classes and import those, no?

Or maybe there is some piece of http://docs.python.org/library/imp.html to use?

[Chris Jerdonek]

(In reply to comment #2)
> No need for "exec".  We can just import the module and then use defs() to grab
> all the classes and import those, no?

Yeah, there should also be ways without exec().

> Or maybe there is some piece of http://docs.python.org/library/imp.html to use?

Here is something else to look at:

http://docs.python.org/library/unittest.html#unittest.TestLoader.loadTestsFromModule

[Chris Jerdonek]

Created attachment 51815 [details]
Proposed patch

I can delete all of the *_unittest files in a subsequent patch.

[Chris Jerdonek]

Created attachment 51816 [details]
Proposed patch 2

Documented the call to unittest.main().

[Adam Barth]

abarth: cjerdonek: so this is good
abarth: but i think you used to be able to specify a single test
abarth: on the command line
abarth: and run just that one
abarth: this will always run them all

[Chris Jerdonek]

(In reply to comment #6)
> abarth: cjerdonek: so this is good
> abarth: but i think you used to be able to specify a single test
> abarth: on the command line
> abarth: and run just that one
> abarth: this will always run them all

cjerdonek: i preserved that but then removed it
[09:58am] abarth: oh, if its easy
[09:58am] abarth: we should support that
[09:59am] cjerdonek: abarth: i'll put it back it then, i didn't think anyone was actually using that

[Chris Jerdonek]

Created attachment 51832 [details]
Proposed patch 3

Reinstated the ability to run a subset of the tests using command-line arguments.

[Adam Barth]

Comment on attachment 51832 [details]
Proposed patch 3

Looks good.  I don't quite understand how we still have the ability to filter tests, but I believe you.

[Chris Jerdonek]

(In reply to comment #9)
> (From update of attachment 51832 [details])
> Looks good.  I don't quite understand how we still have the ability to filter
> tests, but I believe you.

Thanks, Adam!

It's pretty simple.  If sys.argv contains a non-flag argument at the end, then it bypasses auto-detection and does what it normally does:

+        if len(sys_argv) > 1 and not sys_argv[-1].startswith("-"):
+            # Then explicit modules or test names were provided, which
+            # the unittest module is equipped to handle.
+            unittest.main(module=None)
+            # No need to return since unitttest.main() exits.
+
+        # Otherwise, auto-detect all unit tests.
...

[Chris Jerdonek]

Comment on attachment 51832 [details]
Proposed patch 3

Going to land this manually.

[Chris Jerdonek]

Committed via webkit-patch (somewhat): http://trac.webkit.org/changeset/56671

(Didn't quite work out.  Key-chain issues again.  Frustrating because I just tried updating my keychain info using the KeyChain Access app...)


Logging in as [e-mail]...
Bugzilla login failed: Invalid Username Or Password
Reading Keychain for bugs.webkit.org account and password.  Click "Allow" to continue...
Logging in as [e-mail]...
Traceback (most recent call last):
  File "/Users/chris_g4/dev/WebKit/trunk-git/WebKitTools/Scripts/webkit-patch", line 70, in <module>
    main()
  File "/Users/chris_g4/dev/WebKit/trunk-git/WebKitTools/Scripts/webkit-patch", line 65, in main
    WebKitPatch(__file__).main()
  File "/Users/chris_g4/dev/WebKit/trunk-git/WebKitTools/Scripts/webkitpy/tool/multicommandtool.py", line 302, in main
    result = command.check_arguments_and_execute(options, args, self)
  File "/Users/chris_g4/dev/WebKit/trunk-git/WebKitTools/Scripts/webkitpy/tool/multicommandtool.py", line 113, in check_arguments_and_execute
    return self.execute(options, args, tool) or 0
  File "/Users/chris_g4/dev/WebKit/trunk-git/WebKitTools/Scripts/webkitpy/tool/commands/abstractsequencedcommand.py", line 43, in execute
    self._sequence.run_and_handle_errors(tool, options, self._prepare_state(options, args, tool))
  File "/Users/chris_g4/dev/WebKit/trunk-git/WebKitTools/Scripts/webkitpy/tool/commands/stepsequence.py", line 66, in run_and_handle_errors
    self._run(tool, options, state)
  File "/Users/chris_g4/dev/WebKit/trunk-git/WebKitTools/Scripts/webkitpy/tool/commands/stepsequence.py", line 60, in _run
    step(tool, options).run(state)
  File "/Users/chris_g4/dev/WebKit/trunk-git/WebKitTools/Scripts/webkitpy/tool/steps/closebugforlanddiff.py", line 51, in run
    self._tool.bugs.close_bug_as_fixed(bug_id, comment_text)
  File "/Users/chris_g4/dev/WebKit/trunk-git/WebKitTools/Scripts/webkitpy/common/net/bugzilla.py", line 750, in close_bug_as_fixed
    self.authenticate()
  File "/Users/chris_g4/dev/WebKit/trunk-git/WebKitTools/Scripts/webkitpy/common/net/bugzilla.py", line 516, in authenticate
    raise Exception(errorMessage)
Exception: Bugzilla login failed: Invalid Username Or Password