Bug 14468 - JS Number.toPrecision(20) and Number.toPrecision(21) are incorrect
Summary: JS Number.toPrecision(20) and Number.toPrecision(21) are incorrect
Status: RESOLVED DUPLICATE of bug 44745
Alias: None
Product: WebKit
Classification: Unclassified
Component: JavaScriptCore (show other bugs)
Version: 523.x (Safari 3)
Hardware: PC Windows XP
: P2 Normal
Assignee: Nobody
URL:
Keywords: PlatformOnly
Depends on:
Blocks:
 
Reported: 2007-06-29 15:55 PDT by Pam Greene (IRC:pamg)
Modified: 2010-08-27 12:52 PDT (History)
2 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Pam Greene (IRC:pamg) 2007-06-29 15:55:47 PDT 
Number(0.0001234567890123456789012345).toPrecision(20) returns
       0.00012345678901234565000

Number(0.0001234567890123456789012345).toPrecision(21) also returns
       0.000123456789012345650000

Tests failing:
fast/js/kde/Number.html

The problem is that, due to the compiler or FPU or what have you, 1.000000...e+22 * 10.0 is returning 9.999...92e+22 rather than 1.000000...e+23, so intPow10() in kjs/number_object.cpp is wrong and so is toPrecision().

For reference, IE7 gets the result right, and Firefox 2 gives a different wrong answer (0.00012345678901234567130).

A couple of other numerical algorithms I tried give the same erroneous result.  Special-casing 1e+22 is just ugly. I'm tempted to add a precision argument to UString::from(double) and use its string manipulations instead. :/  Any other suggestions?
Comment 1 Robert Blaut 2008-02-15 03:16:19 PST 
Confirmed on the latest Webkit on Windows.
Comment 2 Gavin Barraclough 2010-08-27 12:52:06 PDT 

*** This bug has been marked as a duplicate of bug 44745 ***