RESOLVED DUPLICATE of bug 44745 14468
JS Number.toPrecision(20) and Number.toPrecision(21) are incorrect 
https://bugs.webkit.org/show_bug.cgi?id=14468
Summary JS Number.toPrecision(20) and Number.toPrecision(21) are incorrect
Pam Greene (IRC:pamg)
Reported 2007-06-29 15:55:47 PDT
Number(0.0001234567890123456789012345).toPrecision(20) returns 0.00012345678901234565000 Number(0.0001234567890123456789012345).toPrecision(21) also returns 0.000123456789012345650000 Tests failing: fast/js/kde/Number.html The problem is that, due to the compiler or FPU or what have you, 1.000000...e+22 * 10.0 is returning 9.999...92e+22 rather than 1.000000...e+23, so intPow10() in kjs/number_object.cpp is wrong and so is toPrecision(). For reference, IE7 gets the result right, and Firefox 2 gives a different wrong answer (0.00012345678901234567130). A couple of other numerical algorithms I tried give the same erroneous result. Special-casing 1e+22 is just ugly. I'm tempted to add a precision argument to UString::from(double) and use its string manipulations instead. :/ Any other suggestions?
Attachments
Add attachment proposed patch, testcase, etc.
Robert Blaut
Comment 1 2008-02-15 03:16:19 PST
Confirmed on the latest Webkit on Windows.
Gavin Barraclough
Comment 2 2010-08-27 12:52:06 PDT
*** This bug has been marked as a duplicate of bug 44745 ***
Note You need to log in before you can comment on or make changes to this bug.