Bug 230684 - LayoutTestFinder doesn't return tests in order
Summary: LayoutTestFinder doesn't return tests in order
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Local Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Sam Sneddon [:gsnedders]
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-09-23 04:47 PDT by Sam Sneddon [:gsnedders]
Modified: 2021-09-24 11:50 PDT (History)
5 users (show)

See Also:

Attachments
Patch (5.23 KB, patch)
2021-09-23 11:04 PDT, Sam Sneddon [:gsnedders] 		no flags Details | Formatted Diff | Diff
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Sam Sneddon [:gsnedders] 2021-09-23 04:47:17 PDT 
It is clearly intended that LayoutTestFinder returns the tests in a deterministic order. But it doesn't actually do that: top-level directories appear in an arbitrary order.

Found while rewriting LayoutTestFinder and trying to compare my results against what we currently have.
Comment 1 Sam Sneddon [:gsnedders] 2021-09-23 11:04:44 PDT 
Created attachment 439069 [details]
Patch
Comment 2 Alexey Proskuryakov 2021-09-23 17:51:44 PDT 
Comment on attachment 439069 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=439069&action=review

> Tools/ChangeLog:10
> +        running run-webkit-tests with no arguments ends up running the top-level directories in a

Is this a recent regression? In my experience, accessibility and animations always went first.
Comment 3 Sam Sneddon [:gsnedders] 2021-09-24 05:45:30 PDT 
(In reply to Alexey Proskuryakov from comment #2)
> Comment on attachment 439069 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=439069&action=review
> 
> > Tools/ChangeLog:10
> > +        running run-webkit-tests with no arguments ends up running the top-level directories in a
> 
> Is this a recent regression? In my experience, accessibility and animations
> always went first.

IIRC, it should in practice depend on the order the OS enumerates them (via libc).
Comment 4 EWS 2021-09-24 05:58:48 PDT 
Committed r283037 (242097@main): <https://commits.webkit.org/242097@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 439069 [details].
Comment 5 Radar WebKit Bug Importer 2021-09-24 05:59:19 PDT 
<rdar://problem/83493574>