Maybe this example from "AutoCreate" in the documentation will put you on the right track?
You could use Point i.0 Line and hide the locators with Appearance->None
Manipulate[
Graphics[{Line[u]}, PlotRange -> 2], {{u, {{-1, -1}, {1, 1}}},
Locator, LocatorAutoCreate -> True}]