Yes, k needs to be incremented whether we print something or not, also p needs to go back to 2 for each new k.
k = 2;
While[k < 16, p = 2; While[Mod[k^p, 17] =!= 1, p++];
If[p == 16, Print[k]]; k++]
A simpler way might be
In[2]:= Select[Range[16], FreeQ[Mod[#^Range[15], 17], 1] &]
Out[2]= {3, 5, 6, 7, 10, 11, 12, 14}
or just use the built-in function
In[3]:= PrimitiveRootList[17]
Out[3]= {3, 5, 6, 7, 10, 11, 12, 14}