Bug 177220 - Update test262 tests
Summary: Update test262 tests
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: JavaScriptCore (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Joseph Pecoraro
URL:
Keywords: InRadar
: 174944 (view as bug list)
Depends on:
Blocks:
 
Reported: 2017-09-19 22:20 PDT by Joseph Pecoraro
Modified: 2017-09-27 12:24 PDT (History)
10 users (show)

See Also:


Attachments
[PATCH] Importer Changes (7.51 KB, patch)
2017-09-19 22:32 PDT, Joseph Pecoraro
saam: review+
Details | Formatted Diff | Diff
[PATCH] Everything (24.35 MB, patch)
2017-09-19 22:36 PDT, Joseph Pecoraro
ysuzuki: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Joseph Pecoraro 2017-09-19 22:20:16 PDT
Update test262 tests.

1. Update to today's test262
  - includes ES2017 proposals for things we have implemented (Promise.prototype.finally, etc)
  - includes new tests, updated tests, and corrected tests

2. Improve the importer at the same time
  - importer should better detect upstream git repository for test262-Revision.txt
  - importer should update the JSTest/test262 resources from the test262 checkout
  - importer should take no more than a few seconds to write a yaml file from failures, not 20+ minutes
  - update as needed


Notes:

- The test yaml syntax for Exceptions changed. Example:

test/annexB/language/statements/for-in/var-objectbindingpattern-initializer.js
> /*---
> esid: sec-initializers-in-forin-statement-heads
> description: >
>     for-in initializers with ObjectBindingPattern are always prohibited
> negative:
>   phase: early
>   type: SyntaxError
> ---*/

The `negative` block now includes the both a `type` of exception and the `phase` (early or runtime) it should be thrown. I'm going to just keep our YAML the same and just output the type (SyntaxError here) and ignore the phase, since I think that should be covered by the test itself.

> - path: test262/test/annexB/language/statements/for-in/var-objectbindingpattern-initializer.js
>   cmd: runTest262 :normal, "SyntaxError", ["../../../../../harness/assert.js", "../../../../../harness/sta.js"], []
Comment 1 Joseph Pecoraro 2017-09-19 22:20:51 PDT
*** Bug 174944 has been marked as a duplicate of this bug. ***
Comment 2 Joseph Pecoraro 2017-09-19 22:32:34 PDT
Created attachment 321293 [details]
[PATCH] Importer Changes

These are the changes just to the importer. The rest is basically running the importer a few times to update the yaml, get failures, and re-update the yaml.
Comment 3 Joseph Pecoraro 2017-09-19 22:36:12 PDT
Created attachment 321294 [details]
[PATCH] Everything
Comment 4 Joseph Pecoraro 2017-09-19 22:37:06 PDT
>  - path: test262/test/built-ins/ThrowTypeError/unique-per-realm-non-simple.js
> -  cmd: runTest262 :failDueToOutdatedOrBadTest, "NoException", ["../../../harness/assert.js", "../../../harness/sta.js"], []
> +  cmd: runTest262 :fail, "NoException", ["../../../harness/assert.js", "../../../harness/sta.js"], []

This change is interesting. We seem to have thought the test was incorrect, and after updating the test is failing. So this might be a legit failure, or it might still be that this test262 is wrong. I didn't investigate.
Comment 5 Saam Barati 2017-09-19 23:46:34 PDT
Comment on attachment 321293 [details]
[PATCH] Importer Changes

View in context: https://bugs.webkit.org/attachment.cgi?id=321293&action=review

r=me on both

> Tools/ChangeLog:11
> +        - take seconds instead of 20+ minutes to write yaml with --failures

👍🏼
Comment 6 Yusuke Suzuki 2017-09-20 00:46:00 PDT
Comment on attachment 321294 [details]
[PATCH] Everything

r=me
Comment 7 Joseph Pecoraro 2017-09-20 15:56:00 PDT
ERROR from SVN:
A repository hook failed: Commit blocked by pre-commit hook (exit code 1) with output:

    The following files contain tab characters:

        trunk/JSTests/test262/test/built-ins/Date/prototype/setDate/arg-to-number.js
        trunk/JSTests/test262/test/built-ins/Date/prototype/setFullYear/arg-date-to-number.js
        trunk/JSTests/test262/test/built-ins/Date/prototype/setFullYear/arg-month-to-number.js
        trunk/JSTests/test262/test/built-ins/Date/prototype/setFullYear/arg-year-to-number.js
        trunk/JSTests/test262/test/built-ins/Date/prototype/setHours/arg-hour-to-number.js
        trunk/JSTests/test262/test/built-ins/Date/prototype/setHours/arg-min-to-number.js
        trunk/JSTests/test262/test/built-ins/Date/prototype/setHours/arg-ms-to-number.js
        trunk/JSTests/test262/test/built-ins/Date/prototype/setHours/arg-sec-to-number.js
        trunk/JSTests/test262/test/built-ins/Date/prototype/setMilliseconds/arg-to-number.js
        trunk/JSTests/test262/test/built-ins/Date/prototype/setMinutes/arg-min-to-number.js
        trunk/JSTests/test262/test/built-ins/Date/prototype/setMinutes/arg-ms-to-number.js
        trunk/JSTests/test262/test/built-ins/Date/prototype/setMinutes/arg-sec-to-number.js
        trunk/JSTests/test262/test/built-ins/Date/prototype/setMonth/arg-date-to-number.js
        trunk/JSTests/test262/test/built-ins/Date/prototype/setMonth/arg-month-to-number.js
        trunk/JSTests/test262/test/built-ins/Date/prototype/setSeconds/arg-ms-to-number.js
        trunk/JSTests/test262/test/built-ins/Date/prototype/setSeconds/arg-sec-to-number.js
        trunk/JSTests/test262/test/built-ins/Date/prototype/setTime/arg-to-number.js
        trunk/JSTests/test262/test/language/expressions/class/accessor-name-inst-literal-string-char-escape.js
        trunk/JSTests/test262/test/language/expressions/class/accessor-name-static-literal-string-char-escape.js
        trunk/JSTests/test262/test/language/expressions/object/accessor-name-literal-string-char-escape.js
        trunk/JSTests/test262/test/language/statements/class/accessor-name-inst-literal-string-char-escape.js
        trunk/JSTests/test262/test/language/statements/class/accessor-name-static-literal-string-char-escape.js
Comment 8 Saam Barati 2017-09-20 15:57:03 PDT
I think there might be a way to tell SVN that a directory can have tabs. Alexey, do you know how to do this?
Comment 9 Alexey Proskuryakov 2017-09-20 16:58:15 PDT
I know that we can have allow-tabs on files, but I'm not sure if that's recursive. We can check the source of the pre-commit hook to find out for sure.
Comment 10 Joseph Pecoraro 2017-09-20 20:37:38 PDT
Reminder for the future, this is how I set the "allow-tabs" property on files in an SVN checkout:

    svn propset 'allow-tabs' '' JSTests/test262/test/built-ins/Date/prototype/setDate/arg-to-number.js
    svn propset 'allow-tabs' '' JSTests/test262/test/built-ins/Date/prototype/setFullYear/arg-date-to-number.js
    svn propset 'allow-tabs' '' JSTests/test262/test/built-ins/Date/prototype/setFullYear/arg-month-to-number.js
    svn propset 'allow-tabs' '' JSTests/test262/test/built-ins/Date/prototype/setFullYear/arg-year-to-number.js
    ...
Comment 11 Joseph Pecoraro 2017-09-20 20:38:42 PDT
Landed:
<https://trac.webkit.org/r222311> - the bulk!
<https://trac.webkit.org/r222312> - followup fixing some things my SVN land somehow missed
Comment 12 Radar WebKit Bug Importer 2017-09-27 12:24:44 PDT
<rdar://problem/34693214>