Digit limits

[firebird68]

PRODUCT
Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
(2007-02-22)

QUESTIONS OR COMMENTS
Message: When multiplying 111,111,111 by itself, the Excel result is:
12345678987654300. The real answer when multiplying by hand is :
12345678987654321. Why does Excel round off the last two digits to zero?
Excel appears to rounds off to 15 digits. Why not 16 or more?
--
The Firebird Man

 

[VanguardLH]

Re: Digit limits

firebird68 wrote:

> PRODUCT
> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
> (2007-02-22)
>
> QUESTIONS OR COMMENTS
> Message: When multiplying 111,111,111 by itself, the Excel result is:
> 12345678987654300. The real answer when multiplying by hand is :
> 12345678987654321. Why does Excel round off the last two digits to zero?
> Excel appears to rounds off to 15 digits. Why not 16 or more?

"excel" is not in the name of this newsgroup.

http://www.j-walk.com/SS/excel/usertips/tip032.htm
http://support.microsoft.com/kb/65903
http://excel.tips.net/Pages/T001983_Thoughts_and_Ideas_on_Significant_Digits_in_Excel.html
http://precisioncalc.com/What_is_xlPrecision.html

 

[Ken Blake, MVP]

Re: Digit limits

On Mon, 13 Oct 2008 04:49:01 -0700, firebird68
<firebird68@discussions.microsoft.com> wrote:

> PRODUCT
> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
> (2007-02-22)
>
> QUESTIONS OR COMMENTS
> Message: When multiplying 111,111,111 by itself, the Excel result is:
> 12345678987654300. The real answer when multiplying by hand is :
> 12345678987654321. Why does Excel round off the last two digits to zero?
> Excel appears to rounds off to 15 digits. Why not 16 or more?


You're asking an Excel question in a Windows newsgroup. You are far
more likely to get the help you are looking for if you would ask in an
Excel newsgroup; that's where the Excel experts hang out.

--
Ken Blake, Microsoft MVP - Windows Desktop Experience
Please Reply to the Newsgroup

 

[AlmostBob]

Re: Digit limits

http://www.smokeylake.com/excel/Data.htm
explains how excel stores numbers, and the precision lost after the 15th
(decimal) digit

--
Adaware http://www.lavasoft.de
spybot http://www.safer-networking.org
AVG free antivirus http://www.grisoft.com
Etrust/Vet/CA.online Antivirus scan
http://www3.ca.com/securityadvisor/virusinfo/scan.aspx
Panda online AntiVirus scan http://www.pandasoftware.com/ActiveScan/
Catalog of removal tools (1)
http://www.pandasoftware.com/download/utilities/
Catalog of removal tools (2)
http://www3.ca.com/securityadvisor/newsinfo/collateral.aspx?CID=40387
Blocking Unwanted Parasites with a Hosts file
http://mvps.org/winhelp2002/hosts.htm
links provided as a courtesy, read all instructions on the pages before use

Grateful thanks to the authors and webmasters
_

"firebird68" <firebird68@discussions.microsoft.com> wrote in message
news:44915B4D-2BC9-4CD3-AC45-F23B904E37F2@microsoft.com...
> PRODUCT
> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
> (2007-02-22)
>
> QUESTIONS OR COMMENTS
> Message: When multiplying 111,111,111 by itself, the Excel result is:
> 12345678987654300. The real answer when multiplying by hand is :
> 12345678987654321. Why does Excel round off the last two digits to zero?
> Excel appears to rounds off to 15 digits. Why not 16 or more?
> --
> The Firebird Man

 

[RobertVA]

Re: Digit limits

firebird68 wrote:
> PRODUCT
> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
> (2007-02-22)
>
> QUESTIONS OR COMMENTS
> Message: When multiplying 111,111,111 by itself, the Excel result is:
> 12345678987654300. The real answer when multiplying by hand is :
> 12345678987654321. Why does Excel round off the last two digits to zero?
> Excel appears to rounds off to 15 digits. Why not 16 or more?

In most systems the memory available to store a quantity is limited.
When asked to represent large numbers many software applications store
the number as in a manner resembling scientific notation (like
1.23456789876543 X 10^14 BUT there's a significant possibility the
application uses a power of two instead of a power of ten). Since the
space to store the digits is limited, the number gets rounded off. If
the programmer thinks the matter is important enough the problem can be
eliminated, but that would probably at the expense of slower operation
and/or greatly enlarged storage requirements.

Note that for similar reasons there are numbers that cannot be precisely
stated as fractions. These irrational numbers include pi and the square
root of many prime numbers like two and three.

 

[Larry(LJL269)]

Re: Digit limits

where u really get in trouble is with small #'s that approach 0 (like
1/n as n approaches infinity) which r eventually rounded to 0 & 0's
propogate thru the model or r multiplied by a very large # to get 0.

Solving partial differential eqs numerically which leads to more
deadly A-bomb is 1 example I know of. Big concern in Numerical
Analysis.

Hope that helps. Larry

On Mon, 13 Oct 2008 14:06:36 -0400, RobertVA
<robert_c72athotmail@invalid.com> wrote:

>firebird68 wrote:
>> PRODUCT
>> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
>> (2007-02-22)
>>
>> QUESTIONS OR COMMENTS
>> Message: When multiplying 111,111,111 by itself, the Excel result is:
>> 12345678987654300. The real answer when multiplying by hand is :
>> 12345678987654321. Why does Excel round off the last two digits to zero?
>> Excel appears to rounds off to 15 digits. Why not 16 or more?
>
>In most systems the memory available to store a quantity is limited.
>When asked to represent large numbers many software applications store
>the number as in a manner resembling scientific notation (like
>1.23456789876543 X 10^14 BUT there's a significant possibility the
>application uses a power of two instead of a power of ten). Since the
>space to store the digits is limited, the number gets rounded off. If
>the programmer thinks the matter is important enough the problem can be
>eliminated, but that would probably at the expense of slower operation
>and/or greatly enlarged storage requirements.
>
>Note that for similar reasons there are numbers that cannot be precisely
>stated as fractions. These irrational numbers include pi and the square
>root of many prime numbers like two and three.

----------------------------------------------------------------------

A working unsecure OS is infinitely better than non-working secure OS.
Just spent 1 week cleaning up the mess WUpdate made preventing
hypothetical security problems. http://microscum.com/comsense/

 

[HeyBub]

Re: Digit limits

firebird68 wrote:
> PRODUCT
> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
> (2007-02-22)
>
> QUESTIONS OR COMMENTS
> Message: When multiplying 111,111,111 by itself, the Excel result is:
> 12345678987654300. The real answer when multiplying by hand is :
> 12345678987654321. Why does Excel round off the last two digits to
> zero? Excel appears to rounds off to 15 digits. Why not 16 or more?

Excel uses IEEE double-precision floating point numbers. These numbers are
limited to about 15 significant digits.

As for 16 or more, a limit had to be set somewhere. If the developers had
coded extended precision arithmetic to, oh, say 75 digits of precision, sure
enough somebody would come along and say "why not 76?"