Sorry about that:
Tan[(1/0) /. ComplexInfinity :> Infinity]
now it should work.
But this does not make much sense; it's asking basically to compute Exp[1/0], which, to my mind, makes little sense...
For the ArcTan case use the two-argument version to avoid division by zero:
ArcTan[0, 1]