Bug 15145 - (0.999).toPrecision(1) returns incorrect result.
Summary: (0.999).toPrecision(1) returns incorrect result.
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: JavaScriptCore (show other bugs)
Version: 523.x (Safari 3)
Hardware: Mac OS X 10.4
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-09-04 13:59 PDT by Garth Minette
Modified: 2007-10-14 04:40 PDT (History)
2 users (show)

See Also:


Attachments
Patch to fix Number.toPrecision rounding (3.80 KB, patch)
2007-09-19 19:04 PDT, Jason Foreman
no flags Details | Formatted Diff | Diff
Revised patch to fix Number.toPrecision for 0.999 (3.80 KB, patch)
2007-09-20 10:49 PDT, Jason Foreman
mjs: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Garth Minette 2007-09-04 13:59:16 PDT
In Safari 2.0.4 and also in the latest nightly webkit build, the javascript statement:
    (0.999).toPrecision(1);
will return the incorrect result "0.10" instead of the correct result which would be "1.0"
Comment 1 mitz 2007-09-04 14:15:35 PDT
Assertion failed: (n < intPow10(p)), function callAsFunction, file JavaScriptCore/kjs/number_object.cpp, line 371.
Comment 2 Jason Foreman 2007-09-19 19:04:46 PDT
Created attachment 16331 [details]
Patch to fix Number.toPrecision rounding

This is my first WebKit patch; started with something relatively small and easy.  I welcome any and all input.

The problem seemed to be that when attempting to determine the best value for n, the property n<intPow10(p) was broken and thus strange results were returned.  I hope the way I correct this is acceptable.
Comment 3 Alexey Proskuryakov 2007-09-20 02:00:05 PDT
Comment on attachment 16331 [details]
Patch to fix Number.toPrecision rounding

I haven't really reviewed the patch, but noticed a bugzilla.opendarwin.org URL in the test - please change it to bugs.webkit.org.
Comment 4 Jason Foreman 2007-09-20 10:49:45 PDT
Created attachment 16336 [details]
Revised patch to fix Number.toPrecision for 0.999

Sorry about that old URL, attached is the revised patch.
Comment 5 Maciej Stachowiak 2007-09-29 20:52:10 PDT
Comment on attachment 16336 [details]
Revised patch to fix Number.toPrecision for 0.999

r=me for feature branch
Comment 6 Mark Rowe (bdash) 2007-10-14 04:40:01 PDT
Landed in r26581.