On Sun, Jul 18, 2010 at 2:32 PM, Aaron Lewis <aaron.lewis1989@gmail.com>
wrote:
quoted text
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 07/18/2010 08:04 PM, Floor Terra wrote:
>> On Sun, Jul 18, 2010 at 1:07 PM, Aaron Lewis <aaron.lewis1989@gmail.com>
wrote:
quoted text
>>>> What do you mean by "precision"?
>>>
>>> Kinda of significance digit. For example:
>>> B A number `12.340' , and if say it has 2 digits' precision , then we
>>> consider the `0' is not accurate , while `.34' is accurate.
>>
>> What?!
>
> See:
>
> http://en.wikipedia.org/wiki/Arithmetic_precision

Yes. 12.340 has five significant digits. If you want to store it with
just two significant digits you get 12.

quoted text
> Come on , just for fun , someone asks me if i could print out the
> maximum number that can be stored in a double type , and the *precision*
> of a double type.

The maximum number you can store in a IEEE double is +infinity of course.

A 64 bit double has 52-bit mantissa, together with 1 sign bit it gives
you almost 16 decimal significant digits.
But like other things in life: It's not the size that maters, it's how
you use it. (see math(3) and look for ULP)

quoted text
>
> And the second , well i tried to divide 2 by 3 , see when i reaches 7 (
> should be 0.66....7 ) ,
> may not a clever way.

Be careful converting  your number to decimal if you want to keep
maximum precision.


--
Floor Terra <floort@gmail.com>
www: http://brobding.mine.nu/