You can use $MessagePrePrint to format the message arguments:
$MessagePrePrint = Function[{e}, InputForm[e]]
will make this:
1/0;
give this message:
Power::infy: Infinite expression HoldForm[0^(-1)] encountered.
The HoldForm is required in messages, to avoid an infinite recursion (and InputForm always displays HoldForm as is).
If you don't like the HoldForm, you could convert the HoldForm to a string and remove that:
$MessagePrePrint = Function[{e}, StringTake[ToString[e, InputForm], {10, -2}]]
which gives:
Power::infy: Infinite expression 0^(-1) encountered.
