This is how Mathematica fundamentally works, if you input an exact quantity it will return an exact quantity. Since Erf[2] can not be simply written, it will leave it as is; the simplest way is Erf[2]. If you input a machine-precision number 2.0, it will return a machine precision number. This is therefore expected behaviour.
Compare to e.g. Sin[30 Degree] which gives (exactly) 1/2, but Sin[30.0 Degree] gives 0.5. Hope this is clear. You can force numerical evaluation by doing N[Erf[2]].
