Sorry, Hendrik, I was not aware of your contribution. When I started with mine, I analysed the attached video (marioReversePhi.mp4) that inspired me. I also noticed the flashing colours and even tested alternative sequences other than te one used:
colors =
VideoMapList[
Last@DominantColors[#Image, 2, {"Color", "Coverage"}] &, v];
repeatedColors =
RotateLeft[
FindRepeat[colors[[All, 1]],
SameTest -> (Abs[#1[[1]] - #2[[1]]] < .05 &)], 0]

But I realised that neither the flashing nor the composition of the sequence was the main driver of this illusion. I converted the Mario video to grayscale and the illusion of motion was similar:
VideoFrameMap[ColorConvert[#, "Grayscale"] &, v]

I think that the flashing and coloring may help but the main contribution to the illusion is the edge effect which I tried to explain above.
I am working further on this while trying to answer Sander Huisman about his previous suggestion. I will be back on this very interesting visual effect!
Attachments: