Bug 186399 - test262/Runner.pm: minor updates
Summary: test262/Runner.pm: minor updates
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: JavaScriptCore (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2018-06-07 10:00 PDT by valerie
Modified: 2018-06-12 10:16 PDT (History)
7 users (show)

See Also:


Attachments
Patch (7.09 KB, patch)
2018-06-07 10:12 PDT, valerie
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews202 for win-future (12.82 MB, application/zip)
2018-06-07 12:33 PDT, Build Bot
no flags Details
Patch (7.58 KB, patch)
2018-06-12 07:32 PDT, valerie
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews113 for mac-sierra (3.03 MB, application/zip)
2018-06-12 09:10 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews126 for ios-simulator-wk2 (2.30 MB, application/zip)
2018-06-12 09:22 PDT, Build Bot
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description valerie 2018-06-07 10:00:34 PDT
+++ This bug was initially created as a clone of Bug #184926 +++

Currently, the new Test262-runner only saves the results if called with a --save option. Test262 should generate the results file by default in full runs, while the --save option might be interesting only for creating a new expectations file, listing only failures.
Comment 1 valerie 2018-06-07 10:03:52 PDT
(In reply to valerie from comment #0)
> +++ This bug was initially created as a clone of Bug #184926 +++
> 
> Currently, the new Test262-runner only saves the results if called with a
> --save option. Test262 should generate the results file by default in full
> runs, while the --save option might be interesting only for creating a new
> expectations file, listing only failures.

THIS DESCRIPTION WAS ACCIDENTAL ^

Real description:
- If harness files cannot be found in a custom, supplies test262 path, then default to webkit's test262 harness files.

- Lazily create the test262-results directory (only right before writing to it)

- If an expectation file is supplied via command line, save to the supplied location instead of the default webkit location (JSTest/test262/expectation.yaml).
Comment 2 valerie 2018-06-07 10:12:58 PDT
Created attachment 342183 [details]
Patch
Comment 3 Build Bot 2018-06-07 12:33:17 PDT
Comment on attachment 342183 [details]
Patch

Attachment 342183 [details] did not pass win-ews (win):
Output: http://webkit-queues.webkit.org/results/8061892

New failing tests:
http/tests/security/video-poster-cross-origin-crash2.html
Comment 4 Build Bot 2018-06-07 12:33:28 PDT
Created attachment 342201 [details]
Archive of layout-test-results from ews202 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews202  Port: win-future  Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit
Comment 5 Michael Saboff 2018-06-11 18:09:31 PDT
Comment on attachment 342183 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=342183&action=review

Basically fine after addressing a few issues.

> Tools/ChangeLog:7
> +        - If harness files cannot be found in a custom, supplies
> +        test262 path, then default to webkit's test262 harness files.

I don't quite understand "in a custom, supplies test262 path,"

> Tools/Scripts/test262/Runner.pm:96
>  my $expectationsFile = abs_path("$Bin/../../../JSTests/test262/expectations.yaml");
>  my $configFile = abs_path("$Bin/../../../JSTests/test262/config.yaml");

It seems to make sense to have these paths be based on $webkitTest262Dir.
Comment 6 valerie 2018-06-12 07:32:03 PDT
Created attachment 342534 [details]
Patch
Comment 7 Build Bot 2018-06-12 09:10:33 PDT
Comment on attachment 342534 [details]
Patch

Attachment 342534 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/8147664

New failing tests:
js/dom/JSON-stringify.html
media/video-buffering-allowed.html
Comment 8 Build Bot 2018-06-12 09:10:34 PDT
Created attachment 342545 [details]
Archive of layout-test-results from ews113 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews113  Port: mac-sierra  Platform: Mac OS X 10.12.6
Comment 9 Build Bot 2018-06-12 09:22:22 PDT
Comment on attachment 342534 [details]
Patch

Attachment 342534 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/8147665

New failing tests:
imported/w3c/web-platform-tests/streams/piping/error-propagation-forward.html
Comment 10 Build Bot 2018-06-12 09:22:23 PDT
Created attachment 342549 [details]
Archive of layout-test-results from ews126 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews126  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.4
Comment 11 Michael Saboff 2018-06-12 09:49:58 PDT
Comment on attachment 342534 [details]
Patch

r=me
Comment 12 WebKit Commit Bot 2018-06-12 10:16:56 PDT
Comment on attachment 342534 [details]
Patch

Clearing flags on attachment: 342534

Committed r232755: <https://trac.webkit.org/changeset/232755>
Comment 13 WebKit Commit Bot 2018-06-12 10:16:58 PDT
All reviewed patches have been landed.  Closing bug.