Restrict the movement of a locator either vertically or horizontally

I would like to have a locator list plot with some points that I would like to move only vertically (or horizontally)

Manipulate[
 ListPlot[p, PlotRange -> {{0, Automatic}, {0, Automatic}}],
 	{t, 0, 0.2},
 	{{p, {{1, 1}, {2, 4}, {3, 2}, {4, 5}}}, Locator, Appearance -> ""}]

By default locator moves the points freely to both directions x and y. How can I restrict the movement only to one direction ? For example fix x at 1 and move only vertically or fix y at 4 and move only horizontally ?