I am not 100% sure I understand what you are wanting, but here is a stab at an answer anyway ;-)
First, your graphic doe not have axes, you have removed them, but you do have a frame. perhaps you are wanting the frame ticks to be removed and be left with only a plain rectangle. You can do this by adding the FrameTicks -> None
option.
However, the following example I am doing things slightly differently. The Manipulate in your example is not at all necessary since nothing is being manipulated. So here is a bit of code that does what I think you want: that the coordinates do not appear unless you click on a point in graphic and that when you move your mouse without clicking the coordinates do not change.
DynamicModule[{position = None},
Column[
{
EventHandler[
Framed@ca, {"MouseClicked" :> (position = MousePosition["Graphics"])}],
Dynamic[position]
}]
]
Is this the sort of thing you are looking for?