Bug 79404 – test-webkitpy should run the scm_unittests by default

• Component: Tools / Tests
• Status: RESOLVED DUPLICATE

[Eric Seidel (no email)]

test-webkitpy should run the scm_unittests by default

First we need to fix them, and make them fast. :)

[Dirk Pranke]

+1 :).

[Tim 'mithro' Ansell]

Well, they have been fixed now.

They are still much slower than the majority of the tests, but do complete in under a minute. 

Probably makes sense to enable running of these tests by default on the EWS bots? What do you think?

[Adam Barth]

$ ./Tools/Scripts/test-webkitpy 
Ran 1642 tests in 5.693s                                                                                                                                 

What's the runtime with the scm tests?

[Eric Seidel (no email)]

I can't imagine that we're very far away from making them fast-er.  I dont' know how much work Tim has already done on this.

I do agree that leaving them off is bad.  I've long meant to get back to fixing this bug....

[Tim 'mithro' Ansell]

$ time ./test-webkitpy 
Suppressing most webkitpy logging while running unit tests.
Skipping tests in the following modules or packages because they are really, really, slow:
    webkitpy.common.checkout.scm.scm_unittest
    (https://bugs.webkit.org/show_bug.cgi?id=31818; use --all to include)

Ran 1635 tests in 3.033s                                                                                                                

OK

real    0m3.950s
user    0m18.840s
sys     0m2.130s

$ time ./test-webkitpy --all
Suppressing most webkitpy logging while running unit tests.
Ran 1749 tests in 41.119s                                                                                                               

OK

real    0m42.058s
user    0m58.910s
sys     0m29.420s

$

[Tim 'mithro' Ansell]

My current thought is that they are too slow to run after every change, but fast enough that we should run them before committing the code?

[Dirk Pranke]

I assume they're completing in 40s because they're being run in parallel, right?

I think it's bad that they're off by default. Certainly 40s is much worse than 5, but it's not so bad that we shouldn't run them by default. We should probably retitle most of them as integration tests (the ones that are actually calling out to git and svn) and then they'll fall under the -i flag (which we should enable by default as well, although I'm sure some of them fail now as well) and they can be skipped for people who just want the fast unit tests.

[Adam Barth]

Please don't increase the running time of test-webkitpy by a factor of 10.

If we're going to run these tests by default, they'll need to be reworked so they're not massively slow.

[Tim 'mithro' Ansell]

*** This bug has been marked as a duplicate of bug 107635 ***