I think the problem you're having is that Pause is a blocking operation, which results in it messing up the Manipulate. We can replace it and the While with a Dynamic with UpdateInterval and an If.
Manipulate[
Dynamic[If[run, points = Table[RandomReal[{-1, 1}, 2], 10];
graphic =
ListPlot[points, PlotRange -> {{-1, 1}, {-1, 1}},
PlotLabel -> run];, NotebookClose[window];
Abort[];];, UpdateInterval -> 3], {{run, True}, {True, False}}]
window = CreateDocument[Dynamic[graphic], WindowSize -> {600, 450},
TextAlignment -> Center, ShowCellBracket -> False,
WindowFrame -> "Frameless", WindowFrameElements -> {}];
run
I switched the Manipulate and the window
definition because I thought it makes more sense this way, but you can move them back if you want.