Bug 14468
| Summary: | JS Number.toPrecision(20) and Number.toPrecision(21) are incorrect | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Pam Greene (IRC:pamg) <pam> |
| Component: | JavaScriptCore | Assignee: | Nobody <webkit-unassigned> |
| Status: | RESOLVED DUPLICATE | ||
| Severity: | Normal | CC: | barraclough, webkit |
| Priority: | P2 | Keywords: | PlatformOnly |
| Version: | 523.x (Safari 3) | ||
| Hardware: | PC | ||
| OS: | Windows XP | ||
Pam Greene (IRC:pamg)
Number(0.0001234567890123456789012345).toPrecision(20) returns
0.00012345678901234565000
Number(0.0001234567890123456789012345).toPrecision(21) also returns
0.000123456789012345650000
Tests failing:
fast/js/kde/Number.html
The problem is that, due to the compiler or FPU or what have you, 1.000000...e+22 * 10.0 is returning 9.999...92e+22 rather than 1.000000...e+23, so intPow10() in kjs/number_object.cpp is wrong and so is toPrecision().
For reference, IE7 gets the result right, and Firefox 2 gives a different wrong answer (0.00012345678901234567130).
A couple of other numerical algorithms I tried give the same erroneous result. Special-casing 1e+22 is just ugly. I'm tempted to add a precision argument to UString::from(double) and use its string manipulations instead. :/ Any other suggestions?
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Robert Blaut
Confirmed on the latest Webkit on Windows.
Gavin Barraclough
*** This bug has been marked as a duplicate of bug 44745 ***