Bug 184926 - Auto save the results for Test262
Summary: Auto save the results for Test262
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: JavaScriptCore (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2018-04-24 13:09 PDT by Leo Balter
Modified: 2018-05-01 16:46 PDT (History)
4 users (show)

See Also:


Attachments
Patch (752.26 KB, patch)
2018-04-24 13:56 PDT, Leo Balter
no flags Details | Formatted Diff | Diff
Patch (752.21 KB, patch)
2018-04-25 14:45 PDT, Leo Balter
msaboff: review-
Details | Formatted Diff | Diff
Save results on every run (26.62 KB, patch)
2018-05-01 15:57 PDT, valerie
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Leo Balter 2018-04-24 13:09:25 PDT
Currently, the new Test262-runner only saves the results if called with a --save option. Test262 should generate the results file by default in full runs, while the --save option might be interesting only for creating a new expectations file, listing only failures.
Comment 1 Leo Balter 2018-04-24 13:56:40 PDT
Created attachment 338668 [details]
Patch
Comment 2 Leo Balter 2018-04-24 13:58:37 PDT
The latest patch also includes some minor tweaks to the t262 runner.
Comment 3 Michael Saboff 2018-04-25 13:05:19 PDT
Comment on attachment 338668 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=338668&action=review

r=me with one suggested text change.

> Tools/Scripts/test262/Runner.pm:306
> +        print "\nSaved results in: $expectationsFile\n";

Please change this message to something like "Saved new expectations in :"
Comment 4 Leo Balter 2018-04-25 14:45:09 PDT
Created attachment 338797 [details]
Patch
Comment 5 Michael Saboff 2018-05-01 13:17:32 PDT
Comment on attachment 338797 [details]
Patch

r- for the moving the results and expectations to JSTestt

This should done as two patches.
 - The first patch to save the result.
 - A second patch to move things around.
I would approve the first patch.  I don't like having the test script in Tests/Scripts and the output in JSTests.  I think it does make sense to move both the runner and output files to JSTests.  That may require some rework of the current JSTests/test262 layout.
Comment 6 valerie 2018-05-01 15:57:31 PDT
Created attachment 339234 [details]
Save results on every run
Comment 7 Michael Saboff 2018-05-01 16:00:46 PDT
Comment on attachment 339234 [details]
Save results on every run

r=me
Comment 8 WebKit Commit Bot 2018-05-01 16:38:45 PDT
Comment on attachment 339234 [details]
Save results on every run

Clearing flags on attachment: 339234

Committed r231219: <https://trac.webkit.org/changeset/231219>
Comment 9 WebKit Commit Bot 2018-05-01 16:38:47 PDT
All reviewed patches have been landed.  Closing bug.
Comment 10 Radar WebKit Bug Importer 2018-05-01 16:46:58 PDT
<rdar://problem/39884125>