Bug 65579

Summary: Several Tests of http://test262.ecmascript.org/ fail
Product: WebKit Reporter: lars.sonchocky-helldorf
Component: JavaScriptCoreAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: barraclough, ddkilzer, me, webmaster
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   
URL: http://test262.ecmascript.org/
Bug Depends on: 63398, 70302    
Bug Blocks:    
Attachments:
Description Flags
Results as XML as downloaded from the test site (without detailed description of the failures)
none
Results as RTF as copied from the test site (failures only with detailed description) none

Description lars.sonchocky-helldorf 2011-08-02 16:21:18 PDT
Several Tests of http://test262.ecmascript.org/ fail

What is test262?

test262 is a test suite intended to check agreement between JavaScript implementations and the ECMA-262 Specification (currently 5th Edition). The test suite contains thousands of individual tests, each of which tests some specific requirements of the ECMAScript specification.

currently (WebKit nightly Version 5.1 (6534.50, r92131)) 866 out of 10927 tests fail
Comment 1 Gavin Barraclough 2011-08-02 17:43:01 PDT
We're working on this. :-)

There numerous bugs tracking this, e.g.

https://bugs.webkit.org/show_bug.cgi?id=26382
https://bugs.webkit.org/show_bug.cgi?id=41876
https://bugs.webkit.org/show_bug.cgi?id=61014
Comment 2 Gavin Barraclough 2011-08-08 12:20:59 PDT
*** Bug 41876 has been marked as a duplicate of this bug. ***
Comment 3 Gavin Barraclough 2011-08-12 01:14:39 PDT
*** Bug 36301 has been marked as a duplicate of this bug. ***
Comment 4 Gavin Barraclough 2011-09-22 15:34:19 PDT
Update:
* 51 new tests have been added which put us at 866 out of 10978 fail.
* Function.bind is now implemented, fixing 90 tests, putting us at 776 out of 10978 fail (775 when I fix a trivial issue that I set the number of args incorrectly!).

Looks like the biggest causes of failure are still the inability to delete static properties, and incorrect handling of numeric accessor properties on arrays.
Comment 5 Gavin Barraclough 2011-10-14 18:34:51 PDT
This no longer depends on r61014.  That bug is not fully fixed, but has been fixed in all aspects that would be visible within ES5.
Comment 6 Gavin Barraclough 2011-10-18 01:03:59 PDT
As of r97719 we're down to 568/11016 failing
(or more optimistically, up to 10448 passing!)
Comment 7 lars.sonchocky-helldorf 2011-11-16 13:37:22 PST
One note on the performance: Running those tests in WebKit nightly is way slower than running them in the current Mozilla Firefox 8.0. Firefox seems to be about a magnitude faster than WebKit on that testsuite (same machine of course)

Test Results btw.: Tests To Run: 11029 | Total Tests Ran: 11029 | Pass: 10479 | Fail: 550 | Failed To Load: 0
detailed results attached

Tested with WebKit Nightly Version 5.1.1 (6534.51.22, r100296)
Comment 8 lars.sonchocky-helldorf 2011-11-16 13:39:02 PST
Created attachment 115439 [details]
Results as XML as downloaded from the test site (without detailed description of the failures)
Comment 9 lars.sonchocky-helldorf 2011-11-16 13:40:28 PST
Created attachment 115440 [details]
Results as RTF as copied from the test site (failures only with detailed description)
Comment 10 lars.sonchocky-helldorf 2012-01-11 19:15:01 PST
To be more specific: the testsuite at http://test262.ecmascript.org/ runs quite fast in the beginning it just slows down significantly during runtime.

new results btw.: Tests To Run: 11108 | Total Tests Ran: 11108 | Pass: 10923 | Fail: 185 | Failed To Load: 0
Comment 11 Gavin Barraclough 2012-03-01 16:40:53 PST
Tests To Run: 11563 | Total Tests Ran: 11563 | Pass: 11563 | Fail: 0 | Failed To Load: 0
Fixed in r109461