Have you seen section 2 of this (on "cosmetic" rounding in Excel)?: https://people.eecs.berkeley.edu/~wkahan/Mindless.pdf
Re: The main post
To better understand WL, it should be noted that by default, Mathematica rounds the Front-End output to 6 digits, even though internally, the result has full precision (53 bits, or approximately 15.95 decimal digits). This rounding can be changed with PrintPrecision
:
Style[.1 + .2, PrintPrecision -> 17]
(*
\!\(\*
StyleBox["0.30000000000000004`", (* <-- Note the value in quotation marks *)
StripOnInput->False,
PrintPrecision->17]\)
*)
The Front End displays only the string, 0.30000000000000004
. This is the same result as all IEEE 754 compliant systems compute.
Another point is that Equal
compares with a small relative tolerance. This means that if a
and b
are nonzero numbers that differ by a very small amount, Equal[a, b]
(the same as a == b
) will evaluate to True
. This is what happens in the first test below, but in the second, the right-hand side is zero:
.1 + .2 == .3
(* True *)
.1 + .2 - .3 == 0
(* False *)
You can turn off the tolerance to test exact equality by setting Internal`$EqualTolerance
to zero:
Block[{Internal`$EqualTolerance = 0.},
.1 + .2 == .3]
(* False *)
Another reference:
Goldberg, What Every Computer Scientist Should Know About Floating-Point Arithmetic (1991)