143093 – webkit-patch build-and-test should have an option to iterate over new tests

Bug 143093 - webkit-patch build-and-test should have an option to iterate over new tests

Summary: webkit-patch build-and-test should have an option to iterate over new tests

Status:	RESOLVED FIXED

Alias:	None

Product:	WebKit
Classification:	Unclassified
Component:	Tools / Tests (show other bugs)
Version:	528+ (Nightly build)
Hardware:	Unspecified Unspecified

Importance:	P2 Normal
Assignee:	youenn fablet

URL:
Keywords:

Depends on:	143135
Blocks:
	Show dependency tree / graph

Reported:	2015-03-26 04:35 PDT by youenn fablet
Modified:	2016-01-08 00:37 PST (History)
CC List:	8 users (show)

See Also:

Attachments
Patch (4.24 KB, patch) 2015-03-26 04:40 PDT, youenn fablet	no flags	Details \| Formatted Diff \| Diff
WIP (6.09 KB, patch) 2015-03-27 11:00 PDT, youenn fablet	no flags	Details \| Formatted Diff \| Diff
WIP2 (12.39 KB, patch) 2015-03-30 11:12 PDT, youenn fablet	no flags	Details \| Formatted Diff \| Diff
Patch (16.20 KB, patch) 2015-03-31 02:41 PDT, youenn fablet	no flags	Details \| Formatted Diff \| Diff
Rebasing (17.29 KB, patch) 2016-01-07 06:57 PST, youenn fablet	no flags	Details \| Formatted Diff \| Diff
Patch for landing (17.26 KB, patch) 2016-01-07 23:35 PST, youenn fablet	no flags	Details \| Formatted Diff \| Diff
Show Obsolete (5) View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.

Description youenn fablet 2015-03-26 04:35:15 PDT

As part of enabling limited EWS bots to check new tests, webkit-patch should have the ability to run new tests and report results.
This could be done by adding an option to webkit-patch build-and-test to iterate a number of times on new tests and make a report in case of failures.

Comment 1 youenn fablet 2015-03-26 04:40:34 PDT

Created attachment 249485 [details]
Patch

Comment 2 youenn fablet 2015-03-27 11:00:29 PDT

Created attachment 249585 [details]
WIP

Comment 3 youenn fablet 2015-03-27 11:02:14 PDT

(In reply to comment #2)
> Created attachment 249585 [details]
> WIP

Another try after 143090.
Plan is to add a unit test for the new_or_modified_potential_tests function.
Any feedback appreciated before doing that extra step.

Comment 4 youenn fablet 2015-03-30 11:12:51 PDT

Created attachment 249749 [details]
WIP2

Comment 5 youenn fablet 2015-03-31 02:41:47 PDT

Created attachment 249811 [details]
Patch

Comment 6 youenn fablet 2016-01-07 06:57:43 PST

Created attachment 268450 [details]
Rebasing

Comment 7 youenn fablet 2016-01-07 07:02:20 PST

(In reply to comment #6)
> Created attachment 268450 [details]
> Rebasing

Introduced option (iterate_on_new_tests) may allow gtk/efl/win EWS bots to check new/modified tests at a hopefully small cost.
This option may also be useful in general to check for test flakiness.

Comment 8 Michael Catanzaro 2016-01-07 07:56:09 PST

Comment on attachment 268450 [details]
Rebasing

View in context: https://bugs.webkit.org/attachment.cgi?id=268450&action=review

> Tools/ChangeLog:10
> +        Computationf of new or modified tests is doen by LayoutTestFinder.find_touched_tests

Computationf -> Computation

doen -> done

Comment 9 youenn fablet 2016-01-07 23:35:06 PST

Created attachment 268531 [details]
Patch for landing

Comment 10 youenn fablet 2016-01-07 23:39:23 PST

Thanks for the review.

> > Tools/ChangeLog:10
> > +        Computationf of new or modified tests is doen by LayoutTestFinder.find_touched_tests
> 
> Computationf -> Computation
> 
> doen -> done


Doen... hum done ;)

Comment 11 WebKit Commit Bot 2016-01-08 00:37:05 PST

Comment on attachment 268531 [details]
Patch for landing

Clearing flags on attachment: 268531

Committed r194761: <http://trac.webkit.org/changeset/194761>

Comment 12 WebKit Commit Bot 2016-01-08 00:37:11 PST

All reviewed patches have been landed.  Closing bug.