Bug 66401 - run-api-tests runs tests in a semi-random order
Summary: run-api-tests runs tests in a semi-random order
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Adam Roben (:aroben)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-08-17 13:06 PDT by Adam Roben (:aroben)
Modified: 2011-08-17 14:08 PDT (History)
3 users (show)

See Also:

Attachments
Patch (1.66 KB, patch)
2011-08-17 13:07 PDT, Adam Roben (:aroben) 		ddkilzer: review+
ddkilzer: commit-queue+
Details | Formatted Diff | Diff
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Adam Roben (:aroben) 2011-08-17 13:06:43 PDT 
run-api-tests runs tests in a semi-random order
Comment 1 Adam Roben (:aroben) 2011-08-17 13:07:20 PDT 
Created attachment 104217 [details]
Patch
Comment 2 David Kilzer (:ddkilzer) 2011-08-17 13:09:28 PDT 
Comment on attachment 104217 [details]
Patch

r=me
Comment 3 Dmitry Lomov 2011-08-17 13:15:21 PDT 
Comment on attachment 104217 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=104217&action=review

> Tools/Scripts/run-api-tests:109
> +    for my $suite (sort keys %testsToRun) {

Is it possible to sort testsToRun once and for all instead of sorting the result of keys everytime? (I do not know Perl well enough to answer...)
Comment 4 Adam Roben (:aroben) 2011-08-17 13:18:35 PDT 
Comment on attachment 104217 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=104217&action=review

>> Tools/Scripts/run-api-tests:109
>> +    for my $suite (sort keys %testsToRun) {
> 
> Is it possible to sort testsToRun once and for all instead of sorting the result of keys everytime? (I do not know Perl well enough to answer...)

sort will only run once in this function.
Comment 5 Dmitry Lomov 2011-08-17 13:29:01 PDT 
Comment on attachment 104217 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=104217&action=review

>>> Tools/Scripts/run-api-tests:109
>>> +    for my $suite (sort keys %testsToRun) {
>> 
>> Is it possible to sort testsToRun once and for all instead of sorting the result of keys everytime? (I do not know Perl well enough to answer...)
> 
> sort will only run once in this function.

Sorry I wasn't clear - I meant "populate testsToRun in such a way that all the iterations (both in this function and in runAllTestsInSuite) run in predictable order" (in case somebody adds some other iteration on tests somewhere and forgets to sort).
Comment 6 Adam Roben (:aroben) 2011-08-17 13:51:04 PDT 
Comment on attachment 104217 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=104217&action=review

>>>> Tools/Scripts/run-api-tests:109
>>>> +    for my $suite (sort keys %testsToRun) {
>>> 
>>> Is it possible to sort testsToRun once and for all instead of sorting the result of keys everytime? (I do not know Perl well enough to answer...)
>> 
>> sort will only run once in this function.
> 
> Sorry I wasn't clear - I meant "populate testsToRun in such a way that all the iterations (both in this function and in runAllTestsInSuite) run in predictable order" (in case somebody adds some other iteration on tests somewhere and forgets to sort).

I see. I don't know whether it's possible to sort %testsToRun itself, since it's a hash. We could keep around sorted arrays of suites and test names, but that would just be duplicating data that's already in %testsToRun.
Comment 7 Adam Roben (:aroben) 2011-08-17 14:08:48 PDT 
Committed r93240: <http://trac.webkit.org/changeset/93240>