Bug 15145 – (0.999).toPrecision(1) returns incorrect result.

• Status: RESOLVED
• Resolution: FIXED
• Component: JavaScriptCore

[Garth Minette]

In Safari 2.0.4 and also in the latest nightly webkit build, the javascript statement:
    (0.999).toPrecision(1);
will return the incorrect result "0.10" instead of the correct result which would be "1.0"

Assertion failed: (n < intPow10(p)), function callAsFunction, file JavaScriptCore/kjs/number_object.cpp, line 371.

[Jason Foreman]

Created attachment 16331 [details]
Patch to fix Number.toPrecision rounding

This is my first WebKit patch; started with something relatively small and easy.  I welcome any and all input.

The problem seemed to be that when attempting to determine the best value for n, the property n<intPow10(p) was broken and thus strange results were returned.  I hope the way I correct this is acceptable.

[Alexey Proskuryakov]

Comment on attachment 16331 [details]
Patch to fix Number.toPrecision rounding

I haven't really reviewed the patch, but noticed a bugzilla.opendarwin.org URL in the test - please change it to bugs.webkit.org.

[Jason Foreman]

Created attachment 16336 [details]
Revised patch to fix Number.toPrecision for 0.999

Sorry about that old URL, attached is the revised patch.

[Maciej Stachowiak]

Comment on attachment 16336 [details]
Revised patch to fix Number.toPrecision for 0.999

r=me for feature branch

[Mark Rowe (bdash)]

Landed in r26581.