What are the general techniques for producing smoothly functioning dynamical graphics when a large portion of the graphics is static and only a small portion is dynamic?

To be more specific, suppose I need two plots in a row. They both have different but static colored contour plots. On the first plot I have a Locator. On the second plot I have a simple point with an arrow attached, controlled by the Locator in the first plot. This is all done in a DynamicModule. I know how to do this. The only concern is that if the contour graphic primitives are rather detailed and even though I do not include them within a Dynamic expression, the Locator action can still become sluggish.

Is there a technique for speeding up and smoothing the response of the Locator by handling the static background graphics differently than the active dynamical graphics? My understanding is that there are two processes that take time with graphics, the computation in the kernel of the static primitives for the contour graphics and then the actual rendering of the graphics by the front end. Is that the case and what are the relative times for these actions? If the front end rendering takes much longer then there is a limit to what can be done with detailed graphics. Should LocatorPane with a back plot speed up the response? My experience is that it doesn't.

I may be confused about this entire process, which is why I'm posting the question.

My construction looks something like this:

DynamicModule[{pt={0,0}},
   routine to calculate elements in second plot from locator pt in first plot called by second argument in Dynamic
  Graphics[
    {static graphic elements,
     Dynamic[dynamic graphic elements]}]
]