Bug 185482 – Update Test262 tests through the new import script - 20180509

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Leo Balter]

Update Test262 tests through the new import script - 20180509

[Leo Balter]

Created attachment 339997 [details]
Patch

[Leo Balter]

This patch updates the Test262 files and the respective JSTests/test262/expectations.yaml file with the most recent passes.

This patch helps the report from the Test262 Runner since several tests files started passing from a recent change on WebKit.

[Leo Balter]

There is an error in the expectations file I'm trying to fix right now

[EWS Watchlist]

Comment on attachment 339997 [details]
Patch

Attachment 339997 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/7626712

New failing tests:
http/tests/appcache/abort-cache-onchecking-manifest-404.html

[EWS Watchlist]

Created attachment 340014 [details]
Archive of layout-test-results from ews122 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews122  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.4

[Leo Balter]

Created attachment 340024 [details]
Patch

[Leo Balter]

Created attachment 340026 [details]
Patch

[Michael Saboff]

Comment on attachment 340026 [details]
Patch

r=me

[EWS Watchlist]

Comment on attachment 340026 [details]
Patch

Attachment 340026 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/7628875

New failing tests:
http/tests/appcache/abort-cache-onchecking-manifest-404.html

[EWS Watchlist]

Created attachment 340040 [details]
Archive of layout-test-results from ews100 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews100  Port: mac-sierra  Platform: Mac OS X 10.12.6

[Leo Balter]

seems like something else is breaking. Should I rebase this?

[Michael Saboff]

(In reply to Leo Balter from comment #11)
> seems like something else is breaking. Should I rebase this?

These are unrelated failures.  I just cq+ the patch again.  Let's see if it goes through.  If not, I'll land it manually.

[WebKit Commit Bot]

Comment on attachment 340026 [details]
Patch

Rejecting attachment 340026 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-01', 'apply-attachment', '--no-update', '--non-interactive', 340026, '--port=mac']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit

Last 500 characters of output:
t/language/statements/try/early-catch-duplicates.js
patching file JSTests/test262/test/language/statements/try/early-catch-function.js
patching file JSTests/test262/test/language/statements/try/early-catch-lex.js
patching file JSTests/test262/test/language/statements/try/early-catch-var.js
patching file JSTests/test262/test262-Revision.txt

Failed to run "[u'/Volumes/Data/EWS/WebKit/Tools/Scripts/svn-apply', '--force', '--reviewer', u'Michael Saboff']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Full output: http://webkit-queues.webkit.org/results/7639088

[Leo Balter]

Created attachment 340117 [details]
Patch

[Michael Saboff]

Comment on attachment 340117 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=340117&action=review

r-

> JSTests/ChangeLog:7
> +        Update Test262 tests through the new import script - 20180509
> +
> +        Also update the test262/expecatations.yaml with the recent
> +        imported files.
> +        https://bugs.webkit.org/show_bug.cgi?id=185482

The format of a ChangeLog should be:
    Title
    Bug URL

    Reviewed by ...

    Description

Please move the "Also update..." to after Reviewed By ...

[Leo Balter]

Created attachment 340124 [details]
Patch

[Michael Saboff]

Comment on attachment 340124 [details]
Patch

r=me

[WebKit Commit Bot]

Comment on attachment 340124 [details]
Patch

Clearing flags on attachment: 340124

Committed r231666: <https://trac.webkit.org/changeset/231666>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/40144395>

[Caitlin Potter (:caitp)]

In the future, it would be good to update JSTests/test262.yaml also. I'm unable to run the test262 suite the usual way (run-jsc-stress-tests JSTests/test262.yaml) to update expectations when fixing bugs, due to the large number of deleted files which are still referenced (which cause the python script to crash).

[Leo Balter]

> In the future, it would be good to update JSTests/test262.yaml also. I'm unable to run the test262 suite the usual way (run-jsc-stress-tests JSTests/test262.yaml) to update expectations when fixing bugs, due to the large number of deleted files which are still referenced (which cause the python script to crash).

We introduced a new test262 runner on WebKit already being used by the build bot where the old Test262 runner is still there for a switch back if ever necessary.

Would it be fine for you to use the `test262-runner` instead? It doesn't require any maintenance burden for this legacy test262.yaml file.

You can run it with --help to see some of the extra options and it also generates a results folder with summaries and a full report.

If you have anything else blocking you from using it, please let me understand it and I'll dedicate some time to update the yaml file.