Very interesting.
On the same topic, one thing that I have always missed, and that is fundamental on a 3D engineering environment where we need to navigate through big 3D spaces without getting lost (or seasick...), is dynamic manipulation of the ViewCenter.
I have tried to produce this before, but it always felt too hacky. Have you thought on such application for you code?
Many times I manipulate big 3D objects, and it always feels too difficult inside Mathematica, because the 3D manipulation paradigm was thought for mathematic plots. For instance, most (all?) CAD software (or viewers), use the mouse wheel either to zoom in or out, or to accelerate when in fly mode (which is kind of a wing commander game mode; one software where it works great is on Blender). Since Mathematica is a vertical document, the wheel is locked to scroll along the document. But why can't it have a more CAD oriented behaviour when a Graphics object is selected? Well, sorry, this is already getting out of your topic.
Again, nice work!