You set the significance level (rather than a P-value). The default (as stated in the documentation) is 0.05 but if you want it to be something else such as 0.10, then you can use the following:

data = RandomVariate[NormalDistribution[], 10];
ks = KolmogorovSmirnovTest[data, Automatic, "ShortTestConclusion", SignificanceLevel ->0.10]

How do instead of "reject" --- "ok" and instead of "Do not reject" --- "no"

I need a test result in the Czech language. English is inconvenient for the teacher.

OK. I understand now. (I'm a little slow.) The output is just some text so you could use a brute force approach to make the translation into another variable:

data = BlockRandom[SeedRandom[1]; 
   RandomVariate[NormalDistribution[], 100]];
ks = KolmogorovSmirnovTest[data, NormalDistribution[0, 1.5], "ShortTestConclusion", SignificanceLevel -> .1]
result = If[ks == "Reject", "Ano", "Ne"]