Bug 15145

Summary: (0.999).toPrecision(1) returns incorrect result.
Product: WebKit Reporter: Garth Minette <gminette>
Component: JavaScriptCoreAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: ap, mitz
Priority: P2    
Version: 523.x (Safari 3)   
Hardware: Mac   
OS: OS X 10.4   
Attachments:
Description Flags
Patch to fix Number.toPrecision rounding
none
Revised patch to fix Number.toPrecision for 0.999 mjs: review+

Garth Minette
Reported 2007-09-04 13:59:16 PDT
In Safari 2.0.4 and also in the latest nightly webkit build, the javascript statement: (0.999).toPrecision(1); will return the incorrect result "0.10" instead of the correct result which would be "1.0"
Attachments
Patch to fix Number.toPrecision rounding (3.80 KB, patch)
2007-09-19 19:04 PDT, Jason Foreman 		no flags
DetailsFormatted DiffDiff
Revised patch to fix Number.toPrecision for 0.999 (3.80 KB, patch)
2007-09-20 10:49 PDT, Jason Foreman 		mjs: review+
DetailsFormatted DiffDiff
Show Obsolete (1) View All Add attachment proposed patch, testcase, etc.
mitz
Comment 1 2007-09-04 14:15:35 PDT
Assertion failed: (n < intPow10(p)), function callAsFunction, file JavaScriptCore/kjs/number_object.cpp, line 371.
Jason Foreman
Comment 2 2007-09-19 19:04:46 PDT
Created attachment 16331 [details] Patch to fix Number.toPrecision rounding This is my first WebKit patch; started with something relatively small and easy. I welcome any and all input. The problem seemed to be that when attempting to determine the best value for n, the property n<intPow10(p) was broken and thus strange results were returned. I hope the way I correct this is acceptable.
Alexey Proskuryakov
Comment 3 2007-09-20 02:00:05 PDT
Comment on attachment 16331 [details] Patch to fix Number.toPrecision rounding I haven't really reviewed the patch, but noticed a bugzilla.opendarwin.org URL in the test - please change it to bugs.webkit.org.
Jason Foreman
Comment 4 2007-09-20 10:49:45 PDT
Created attachment 16336 [details] Revised patch to fix Number.toPrecision for 0.999 Sorry about that old URL, attached is the revised patch.
Maciej Stachowiak
Comment 5 2007-09-29 20:52:10 PDT
Comment on attachment 16336 [details] Revised patch to fix Number.toPrecision for 0.999 r=me for feature branch
Mark Rowe (bdash)
Comment 6 2007-10-14 04:40:01 PDT
Landed in r26581.
Note You need to log in before you can comment on or make changes to this bug.