Bug 147607 - [INTL] Implement String.prototype.localeCompare in ECMA-402
Summary: [INTL] Implement String.prototype.localeCompare in ECMA-402
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: JavaScriptCore (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Filip Pizlo
URL:
Keywords: InRadar
Depends on: 155471
Blocks: 90906
  Show dependency treegraph
 
Reported: 2015-08-03 17:28 PDT by Andy VanWagoner
Modified: 2016-04-24 10:15 PDT (History)
14 users (show)

See Also:


Attachments
Patch (20.56 KB, patch)
2015-12-19 10:21 PST, Andy VanWagoner
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews103 for mac-yosemite (767.13 KB, application/zip)
2015-12-19 11:09 PST, Build Bot
no flags Details
Archive of layout-test-results from ews107 for mac-yosemite-wk2 (965.50 KB, application/zip)
2015-12-19 11:12 PST, Build Bot
no flags Details
Patch (20.28 KB, patch)
2015-12-19 11:14 PST, Andy VanWagoner
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews103 for mac-yosemite (978.38 KB, application/zip)
2015-12-19 11:40 PST, Build Bot
no flags Details
Archive of layout-test-results from ews105 for mac-yosemite-wk2 (763.28 KB, application/zip)
2015-12-19 12:05 PST, Build Bot
no flags Details
Archive of layout-test-results from ews116 for mac-yosemite (789.95 KB, application/zip)
2015-12-19 12:15 PST, Build Bot
no flags Details
Patch (22.72 KB, patch)
2015-12-19 12:27 PST, Andy VanWagoner
no flags Details | Formatted Diff | Diff
Patch (23.09 KB, patch)
2015-12-23 10:16 PST, Andy VanWagoner
no flags Details | Formatted Diff | Diff
work in progress (33.53 KB, patch)
2016-04-23 16:20 PDT, Filip Pizlo
no flags Details | Formatted Diff | Diff
the patch (41.39 KB, patch)
2016-04-23 20:50 PDT, Filip Pizlo
buildbot: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from ews116 for mac-yosemite (962.97 KB, application/zip)
2016-04-23 21:59 PDT, Build Bot
no flags Details
the patch (41.67 KB, patch)
2016-04-23 22:14 PDT, Filip Pizlo
darin: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Andy VanWagoner 2015-08-03 17:28:33 PDT
Implement ECMA-402 2.0 13.1.1 String.prototype.localeCompare (that [, locales [, options ]])
http://ecma-international.org/publications/standards/Ecma-402.htm

This definition supersedes the definition provided in ES2015, 21.1.3.10.
Comment 1 Timothy Hatcher 2015-08-27 11:17:25 PDT
<rdar://problem/15455101>
Comment 2 Radar WebKit Bug Importer 2015-08-27 11:18:04 PDT
<rdar://problem/22459074>
Comment 3 Andy VanWagoner 2015-12-19 10:21:06 PST
Created attachment 267691 [details]
Patch
Comment 4 Build Bot 2015-12-19 11:08:58 PST
Comment on attachment 267691 [details]
Patch

Attachment 267691 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/580889

New failing tests:
sputnik/Conformance/15_Native_Objects/15.5_String/15.5.4/15.5.4.9_String.prototype.localeCompare/S15.5.4.9_3.html
js/kde/StringObject.html
js/dom/string-prototype-properties.html
Comment 5 Build Bot 2015-12-19 11:09:02 PST
Created attachment 267692 [details]
Archive of layout-test-results from ews103 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 6 Build Bot 2015-12-19 11:12:07 PST
Comment on attachment 267691 [details]
Patch

Attachment 267691 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/580893

New failing tests:
sputnik/Conformance/15_Native_Objects/15.5_String/15.5.4/15.5.4.9_String.prototype.localeCompare/S15.5.4.9_3.html
js/kde/StringObject.html
js/dom/string-prototype-properties.html
Comment 7 Build Bot 2015-12-19 11:12:10 PST
Created attachment 267694 [details]
Archive of layout-test-results from ews107 for mac-yosemite-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews107  Port: mac-yosemite-wk2  Platform: Mac OS X 10.10.5
Comment 8 Andy VanWagoner 2015-12-19 11:14:27 PST
Created attachment 267695 [details]
Patch
Comment 9 Build Bot 2015-12-19 11:40:20 PST
Comment on attachment 267695 [details]
Patch

Attachment 267695 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/580999

New failing tests:
js/dom/string-prototype-properties.html
Comment 10 Build Bot 2015-12-19 11:40:24 PST
Created attachment 267697 [details]
Archive of layout-test-results from ews103 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 11 Build Bot 2015-12-19 12:05:51 PST
Comment on attachment 267695 [details]
Patch

Attachment 267695 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/581055

New failing tests:
js/dom/string-prototype-properties.html
Comment 12 Build Bot 2015-12-19 12:05:54 PST
Created attachment 267698 [details]
Archive of layout-test-results from ews105 for mac-yosemite-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews105  Port: mac-yosemite-wk2  Platform: Mac OS X 10.10.5
Comment 13 Build Bot 2015-12-19 12:15:56 PST
Comment on attachment 267695 [details]
Patch

Attachment 267695 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/581074

New failing tests:
js/dom/string-prototype-properties.html
Comment 14 Build Bot 2015-12-19 12:15:59 PST
Created attachment 267699 [details]
Archive of layout-test-results from ews116 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews116  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 15 Andy VanWagoner 2015-12-19 12:27:51 PST
Created attachment 267700 [details]
Patch
Comment 16 WebKit Commit Bot 2015-12-21 02:04:02 PST
Comment on attachment 267700 [details]
Patch

Clearing flags on attachment: 267700

Committed r194328: <http://trac.webkit.org/changeset/194328>
Comment 17 WebKit Commit Bot 2015-12-21 02:04:06 PST
All reviewed patches have been landed.  Closing bug.
Comment 18 Ryan Haddad 2015-12-21 08:40:15 PST
The following JSC tests are failing after this change:

** The following JSC stress test failures have been introduced:
	cdjs-tests.yaml/main.js.default
	cdjs-tests.yaml/main.js.ftl
	cdjs-tests.yaml/main.js.ftl-eager-no-cjit
	cdjs-tests.yaml/main.js.ftl-no-cjit

<https://build.webkit.org/builders/Apple%20Yosemite%20Debug%20JSC%20%28Tests%29/builds/4406>
<https://build.webkit.org/builders/Apple%20Yosemite%2032-bit%20JSC%20%28BuildAndTest%29/builds/7398>
<https://build.webkit.org/builders/Apple%20El%20Capitan%20Debug%20JSC%20%28Tests%29/builds/1101>
<https://build.webkit.org/builders/Apple%20El%20Capitan%2032-bit%20JSC%20%28BuildAndTest%29/builds/943>
Comment 19 Ryan Haddad 2015-12-21 10:21:24 PST
Reverted r194328 for reason:

This change appears to have caused failures in JSC tests

Committed r194332: <http://trac.webkit.org/changeset/194332>
Comment 20 Andy VanWagoner 2015-12-23 10:16:29 PST
Created attachment 267843 [details]
Patch
Comment 21 WebKit Commit Bot 2015-12-23 12:03:01 PST
Comment on attachment 267843 [details]
Patch

Clearing flags on attachment: 267843

Committed r194394: <http://trac.webkit.org/changeset/194394>
Comment 22 WebKit Commit Bot 2015-12-23 12:03:08 PST
All reviewed patches have been landed.  Closing bug.
Comment 23 Filip Pizlo 2016-03-14 14:24:06 PDT
This patch is a 2x slow-down on CDjs.

Did you run any performance tests before landing this patch?  We usually have a policy that nothing lands without performance evaluation, especially if it's in code that's important to benchmarks.
Comment 24 Filip Pizlo 2016-03-14 14:44:34 PDT
Comment on attachment 267843 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=267843&action=review

> Source/JavaScriptCore/ChangeLog:11
> +        Add localeCompare in builtin JavaScript that delegates comparing to Intl.Collator.
> +        Keep existing native implementation for use if INTL flag is disabled.
> +        For the common case where no locale or options are specified, avoid creating
> +        a new collator and just use the prototype which is initialized with the defaults.

Your changelog implies that you have not run performance, and yet this patch affects the behavior and performance of a critical piece of VM functionality.

Please make sure that you run comprehensive performance tests whenever making changes to important things in the VM.  It's customary to note in the changelog which benchmarks were run and what the results were.

> Source/JavaScriptCore/builtins/StringPrototype.js:28
> +function localeCompare(that/*, locales, options */)

Please make "locales" and "options" be actual arguments to this function.

> Source/JavaScriptCore/builtins/StringPrototype.js:40
> +    if (this === null)
> +        throw new @TypeError("String.prototype.localeCompare requires that |this| not be null");
> +    
> +    if (this === undefined)
> +        throw new @TypeError("String.prototype.localeCompare requires that |this| not be undefined");

This should be one branch rather than two.  For example:

// Catch both null and undefined.
if (this == null) {
    if (this === null)
        throw new @TypeError("String.prototype.localeCompare requires that |this| not be null");
    throw new @TypeError("String.prototype.localeCompare requires that |this| not be undefined");
}

> Source/JavaScriptCore/builtins/StringPrototype.js:51
> +    if (arguments[1] === undefined && arguments[2] === undefined)

It's really bad that this is using the "arguments" object.  Please instead use the actual "locales" and "options" arguments by name.  That avoids having to allocate the arguments object for no good reason.

> Source/JavaScriptCore/builtins/StringPrototype.js:52
> +        return @Collator.prototype.compare(thisString, thatString);

This is probably incorrect behavior.  The spec probably means the collator's original prototype, not the prototype it happens to have now.

> Source/JavaScriptCore/builtins/StringPrototype.js:59
> +    // 6. Let collator be Construct(%Collator%, «locales, options»).
> +    // 7. ReturnIfAbrupt(collator).
> +    var collator = new @Collator(arguments[1], arguments[2]);
> +
> +    // 8. Return CompareStrings(collator, S, That).
> +    return collator.compare(thisString, thatString);

It's really bad that you're allocating a new Collator object every time.  This suggest that this should never have been a builtin.  It should be a native function instead.
Comment 25 Filip Pizlo 2016-03-14 16:31:45 PDT
Rolled out in http://trac.webkit.org/changeset/198171 because of huge performance regressions.

I think that these behavioral changes can probably be implemented without performance regressions.  I'm happy to help with this, whenever someone decides to land this again.
Comment 26 Filip Pizlo 2016-03-14 17:26:10 PDT
I'll look at relanding this after I'm done with some other regressions...
Comment 27 Filip Pizlo 2016-04-23 16:20:09 PDT
Created attachment 277170 [details]
work in progress

This is a combination of Andy's patch and some work to try to reduce the cost of the localeCompare() builtin.  I think I know how to do it.
Comment 28 Filip Pizlo 2016-04-23 20:50:44 PDT
Created attachment 277176 [details]
the patch
Comment 29 Build Bot 2016-04-23 21:59:42 PDT
Comment on attachment 277176 [details]
the patch

Attachment 277176 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/1210666

New failing tests:
js/regress/locale-compare.html
Comment 30 Build Bot 2016-04-23 21:59:46 PDT
Created attachment 277181 [details]
Archive of layout-test-results from ews116 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews116  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 31 Filip Pizlo 2016-04-23 22:14:45 PDT
Created attachment 277182 [details]
the patch

Fixes some DFG validation failures.
Comment 32 Filip Pizlo 2016-04-24 10:05:44 PDT
Landed in http://trac.webkit.org/changeset/199967
Comment 33 Filip Pizlo 2016-04-24 10:15:32 PDT
(In reply to comment #24)
> Comment on attachment 267843 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=267843&action=review
> 
> > Source/JavaScriptCore/ChangeLog:11
> > +        Add localeCompare in builtin JavaScript that delegates comparing to Intl.Collator.
> > +        Keep existing native implementation for use if INTL flag is disabled.
> > +        For the common case where no locale or options are specified, avoid creating
> > +        a new collator and just use the prototype which is initialized with the defaults.
> 
> Your changelog implies that you have not run performance, and yet this patch
> affects the behavior and performance of a critical piece of VM functionality.
> 
> Please make sure that you run comprehensive performance tests whenever
> making changes to important things in the VM.  It's customary to note in the
> changelog which benchmarks were run and what the results were.
> 
> > Source/JavaScriptCore/builtins/StringPrototype.js:28
> > +function localeCompare(that/*, locales, options */)
> 
> Please make "locales" and "options" be actual arguments to this function.

Turns out we probably don't want to do that, at least not without more builtins parsing support to be able to specify that the .length property reports "1" even though we have three arguments.

> 
> > Source/JavaScriptCore/builtins/StringPrototype.js:40
> > +    if (this === null)
> > +        throw new @TypeError("String.prototype.localeCompare requires that |this| not be null");
> > +    
> > +    if (this === undefined)
> > +        throw new @TypeError("String.prototype.localeCompare requires that |this| not be undefined");
> 
> This should be one branch rather than two.  For example:
> 
> // Catch both null and undefined.
> if (this == null) {
>     if (this === null)
>         throw new @TypeError("String.prototype.localeCompare requires that
> |this| not be null");
>     throw new @TypeError("String.prototype.localeCompare requires that
> |this| not be undefined");
> }

This is still a good idea, but I haven't done it yet.

> 
> > Source/JavaScriptCore/builtins/StringPrototype.js:51
> > +    if (arguments[1] === undefined && arguments[2] === undefined)
> 
> It's really bad that this is using the "arguments" object.  Please instead
> use the actual "locales" and "options" arguments by name.  That avoids
> having to allocate the arguments object for no good reason.

I fixed some FTL bugs that were preventing escape analysis from removing the allocation.  But I think that in future, we should find a way to write such code without using 'arguments', since 'arguments' object optimizations are gated on a lot of stuff.

> 
> > Source/JavaScriptCore/builtins/StringPrototype.js:52
> > +        return @Collator.prototype.compare(thisString, thatString);
> 
> This is probably incorrect behavior.  The spec probably means the collator's
> original prototype, not the prototype it happens to have now.
> 
> > Source/JavaScriptCore/builtins/StringPrototype.js:59
> > +    // 6. Let collator be Construct(%Collator%, «locales, options»).
> > +    // 7. ReturnIfAbrupt(collator).
> > +    var collator = new @Collator(arguments[1], arguments[2]);
> > +
> > +    // 8. Return CompareStrings(collator, S, That).
> > +    return collator.compare(thisString, thatString);
> 
> It's really bad that you're allocating a new Collator object every time. 
> This suggest that this should never have been a builtin.  It should be a
> native function instead.

I don't know what to do about this yet.  I think that this has to be a builtin so long as the "Collator.prototype.compare()" call is there, since this needs a lot of caching to be fast.  I still want to figure out how to avoid the @Collator allocation on this path, but I don't have a benchmark to measure that path yet.