I am not shure, Henrik, if I understand you or you understood me. Where is that static Background in your example?
I added a randomly rotated LightBlue rectangle as such. Of course it is recalculated with each step, i.e. move of the locator:
grgr = GraphicsGroup[{Red, Disk[], Blue, Rectangle[{.5, .5}], Green,
Sphere[{.5, .3}, 1.4]}]; Manipulate[
Graphics[{LightBlue,
GeometricTransformation[Rectangle[{-3, -3}, {3, 3}],
RotationTransform[RandomReal[{0, Pi/2}]]],
Translate[grgr, {xy}]}, PlotRange -> {{-4, 4}, {-4, 4}},
Axes -> True], {{xy, {1, 1}}, Locator}]