test262/Runner.pm: minor updates

• Component: JavaScriptCore
• Status: RESOLVED FIXED

[valerie]

+++ This bug was initially created as a clone of Bug #184926 +++

Currently, the new Test262-runner only saves the results if called with a --save option. Test262 should generate the results file by default in full runs, while the --save option might be interesting only for creating a new expectations file, listing only failures.

[valerie]

(In reply to valerie from comment #0)
> +++ This bug was initially created as a clone of Bug #184926 +++
> 
> Currently, the new Test262-runner only saves the results if called with a
> --save option. Test262 should generate the results file by default in full
> runs, while the --save option might be interesting only for creating a new
> expectations file, listing only failures.

THIS DESCRIPTION WAS ACCIDENTAL ^

Real description:
- If harness files cannot be found in a custom, supplies test262 path, then default to webkit's test262 harness files.

- Lazily create the test262-results directory (only right before writing to it)

- If an expectation file is supplied via command line, save to the supplied location instead of the default webkit location (JSTest/test262/expectation.yaml).

[valerie]

Created attachment 342183 [details]
Patch

[EWS Watchlist]

Comment on attachment 342183 [details]
Patch

Attachment 342183 [details] did not pass win-ews (win):
Output: http://webkit-queues.webkit.org/results/8061892

New failing tests:
http/tests/security/video-poster-cross-origin-crash2.html

[EWS Watchlist]

Created attachment 342201 [details]
Archive of layout-test-results from ews202 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews202  Port: win-future  Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit

[Michael Saboff]

Comment on attachment 342183 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=342183&action=review

Basically fine after addressing a few issues.

> Tools/ChangeLog:7
> +        - If harness files cannot be found in a custom, supplies
> +        test262 path, then default to webkit's test262 harness files.

I don't quite understand "in a custom, supplies test262 path,"

> Tools/Scripts/test262/Runner.pm:96
>  my $expectationsFile = abs_path("$Bin/../../../JSTests/test262/expectations.yaml");
>  my $configFile = abs_path("$Bin/../../../JSTests/test262/config.yaml");

It seems to make sense to have these paths be based on $webkitTest262Dir.

[valerie]

Created attachment 342534 [details]
Patch

[EWS Watchlist]

Comment on attachment 342534 [details]
Patch

Attachment 342534 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/8147664

New failing tests:
js/dom/JSON-stringify.html
media/video-buffering-allowed.html

[EWS Watchlist]

Created attachment 342545 [details]
Archive of layout-test-results from ews113 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews113  Port: mac-sierra  Platform: Mac OS X 10.12.6

[EWS Watchlist]

Comment on attachment 342534 [details]
Patch

Attachment 342534 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/8147665

New failing tests:
imported/w3c/web-platform-tests/streams/piping/error-propagation-forward.html

[EWS Watchlist]

Created attachment 342549 [details]
Archive of layout-test-results from ews126 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews126  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.4

[Michael Saboff]

Comment on attachment 342534 [details]
Patch

r=me

[WebKit Commit Bot]

Comment on attachment 342534 [details]
Patch

Clearing flags on attachment: 342534

Committed r232755: <https://trac.webkit.org/changeset/232755>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.