1) The significance level in a hypothesis test has no effect on the p-value. The significance level is the threshold for interpreting the p-value (deciding whether the p-value indicates a statistically significant departure from the null hypothesis). The SignificanceLevel option in the TTest function has an effect only if the interpretation is included in the output, as in:
In[]:= TTest[{1, 2, 3, 4, 5}, 1, {"PValue", "TestConclusion"}]
Out[701]= {0.0474207,
The null hypothesis that the mean of the population is equal to 1 is rejected at the 5 percent level based on the T test.}
In[]:= TTest[{1, 2, 3, 4, 5}, 1, {"PValue", "TestConclusion"}, SignificanceLevel -> 0.01]
Out[702]= {0.0474207,
The null hypothesis that the mean of the population is equal to 1 is not rejected at the 1 percent level based on the T test.}
2) Unlike the sample that was used in the video, the mean of the sample generated by data1 = RandomVariate[NormalDistribution[1000, 0.254], 2500] is very close to 1000, and the sample size is 100 times bigger than the sample used in the video. Of those two differences, the most important difference here is that the sample mean is very close to 1000, which is the population mean under the null hypothesis. TTest[data1, 1000] will return a large p-value because the mean of the sample is (by construction) very close to 1000. Replacing 0.254 by 0.0254 or 0.00254 makes the population standard deviation smaller, but since the mean of the sample is still 1000, the p-values will be big.
It is easier to see the effect of population standard deviation on the p-value by using samples from populations for which the mean is not equal the the mean under the null hypothesis, such as:
In[]:= data1 = RandomVariate[NormalDistribution[1001, 3], 10];
TTest[data1, 1000]
Out[]= 0.760372
In[]:= data2 = RandomVariate[NormalDistribution[1001, 1], 10];
TTest[data2, 1000]
Out[]= 0.0812006
In[]:= data3 = RandomVariate[NormalDistribution[1001, 0.5], 10];
TTest[data3, 1000]
Out[]= 0.0000260985
