Bug 66401

Summary: run-api-tests runs tests in a semi-random order
Product: WebKit Reporter: Adam Roben (:aroben) <aroben>
Component: Tools / TestsAssignee: Adam Roben (:aroben) <aroben>
Status: RESOLVED FIXED    
Severity: Normal CC: ddkilzer, dslomov, sam
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch ddkilzer: review+, ddkilzer: commit-queue+

Description Adam Roben (:aroben) 2011-08-17 13:06:43 PDT
run-api-tests runs tests in a semi-random order
Comment 1 Adam Roben (:aroben) 2011-08-17 13:07:20 PDT
Created attachment 104217 [details]
Patch
Comment 2 David Kilzer (:ddkilzer) 2011-08-17 13:09:28 PDT
Comment on attachment 104217 [details]
Patch

r=me
Comment 3 Dmitry Lomov 2011-08-17 13:15:21 PDT
Comment on attachment 104217 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=104217&action=review

> Tools/Scripts/run-api-tests:109
> +    for my $suite (sort keys %testsToRun) {

Is it possible to sort testsToRun once and for all instead of sorting the result of keys everytime? (I do not know Perl well enough to answer...)
Comment 4 Adam Roben (:aroben) 2011-08-17 13:18:35 PDT
Comment on attachment 104217 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=104217&action=review

>> Tools/Scripts/run-api-tests:109
>> +    for my $suite (sort keys %testsToRun) {
> 
> Is it possible to sort testsToRun once and for all instead of sorting the result of keys everytime? (I do not know Perl well enough to answer...)

sort will only run once in this function.
Comment 5 Dmitry Lomov 2011-08-17 13:29:01 PDT
Comment on attachment 104217 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=104217&action=review

>>> Tools/Scripts/run-api-tests:109
>>> +    for my $suite (sort keys %testsToRun) {
>> 
>> Is it possible to sort testsToRun once and for all instead of sorting the result of keys everytime? (I do not know Perl well enough to answer...)
> 
> sort will only run once in this function.

Sorry I wasn't clear - I meant "populate testsToRun in such a way that all the iterations (both in this function and in runAllTestsInSuite) run in predictable order" (in case somebody adds some other iteration on tests somewhere and forgets to sort).
Comment 6 Adam Roben (:aroben) 2011-08-17 13:51:04 PDT
Comment on attachment 104217 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=104217&action=review

>>>> Tools/Scripts/run-api-tests:109
>>>> +    for my $suite (sort keys %testsToRun) {
>>> 
>>> Is it possible to sort testsToRun once and for all instead of sorting the result of keys everytime? (I do not know Perl well enough to answer...)
>> 
>> sort will only run once in this function.
> 
> Sorry I wasn't clear - I meant "populate testsToRun in such a way that all the iterations (both in this function and in runAllTestsInSuite) run in predictable order" (in case somebody adds some other iteration on tests somewhere and forgets to sort).

I see. I don't know whether it's possible to sort %testsToRun itself, since it's a hash. We could keep around sorted arrays of suites and test names, but that would just be duplicating data that's already in %testsToRun.
Comment 7 Adam Roben (:aroben) 2011-08-17 14:08:48 PDT
Committed r93240: <http://trac.webkit.org/changeset/93240>