Yes, with weighting function PDF[NormalDistribution[], #] & and data values around 80, the weights are values of the probability density function of the standard normal distribution around 80 standard deviations above the mean, so the weights are smaller (around 10^-1400) than the smallest possible machine number (which is about 10^-323). NormalDistribution[] refers to the standard normal distribution, which is the normal distribution with a mean of zero and a standard deviation of 1, so NormalDistribution is equivalent to NormalDistribution[0,1]. I don't know the limitations of the EstimatedDistribution function, but the error messages suggest that it is not designed to deal with weights that are that small. I'm not sure if that is what was intended here, but the following worked when I tried it:

sampleMean = Mean[wdAllFlat];

sampleStandardDeviation = StandardDeviation[wdAllFlat];

weightedData = WeightedData[wdAllFlat, 
   PDF[NormalDistribution[sampleMean, sampleStandardDeviation], #] &];

EstimatedDistribution[weightedData, NormalDistribution[\[Mu], \[Sigma]]]

1) The significance level in a hypothesis test has no effect on the p-value. The significance level is the threshold for interpreting the p-value (deciding whether the p-value indicates a statistically significant departure from the null hypothesis). The SignificanceLevel option in the TTest function has an effect only if the interpretation is included in the output, as in:

In[]:= TTest[{1, 2, 3, 4, 5}, 1, {"PValue", "TestConclusion"}]

Out[701]= {0.0474207, 
  The null hypothesis that the mean of the population is equal to 1 is rejected at the 5 percent level based on the T test.}

In[]:= TTest[{1, 2, 3, 4, 5}, 1, {"PValue", "TestConclusion"}, SignificanceLevel -> 0.01]

Out[702]= {0.0474207, 
The null hypothesis that the mean of the population is equal to 1 is not rejected at the 1 percent level based on the T test.}

2) Unlike the sample that was used in the video, the mean of the sample generated by data1 = RandomVariate[NormalDistribution[1000, 0.254], 2500] is very close to 1000, and the sample size is 100 times bigger than the sample used in the video. Of those two differences, the most important difference here is that the sample mean is very close to 1000, which is the population mean under the null hypothesis. TTest[data1, 1000] will return a large p-value because the mean of the sample is (by construction) very close to 1000. Replacing 0.254 by 0.0254 or 0.00254 makes the population standard deviation smaller, but since the mean of the sample is still 1000, the p-values will be big.

It is easier to see the effect of population standard deviation on the p-value by using samples from populations for which the mean is not equal the the mean under the null hypothesis, such as:

In[]:= data1 = RandomVariate[NormalDistribution[1001, 3], 10];
TTest[data1, 1000]

Out[]= 0.760372

In[]:= data2 = RandomVariate[NormalDistribution[1001, 1], 10];
TTest[data2, 1000]

Out[]= 0.0812006

In[]:= data3 = RandomVariate[NormalDistribution[1001, 0.5], 10];
TTest[data3, 1000]

Out[]= 0.0000260985

Thanks again. Some additional experiments with data where the mean moves away from 1000 illustrate the interaction between PValue, the significance level, and the conclusion of the TTest with respect to rejection of the null hypothesis.

Attachments:

Example_pValue_S...nb

Please post the link to the course framework on this community site. The chat pane does not appear in the recording of the course so if I miss that lecture I won't see the link. Thanks!

Thanks. Understood.

Hi;

I am interested in finding the value of "x" in a probability when I know the probability in which I am interested. Intuitively one would assume that the Solve function should produce the answer by simply solving for x - see below. However, Solve, SolveValue or Reduce does not return the desired results. In some situations, I know the probability in which I am interested but I do not know the value of x that gives that probability without a lot of trial and error, so I hope someone could point me in the right direction.

Thanks, Mitch Sandlin

Solve[0.6 == Probability[x, x [Distributed] NormalDistribution[998, 202]], x]

Hi Juan;

Thanks so much. It was actually the InverseCDF function that performed the correct calculation.

Mitch Sandlin

We will share links to quizzes and final exam during the DSG.

Hi Marvin,

The reason is that Show takes PlotRange from its first argument (Histogram) and you will notice that the second argument (ListLinePlot) has a very different PlotRange. Why???

Because, unexpectedly, MovingAverage performs a unit conversion from Celcius to Kelvin.

MovingAverage[{Quantity[22, "DegreesCelsius"], Quantity[24, "DegreesCelsius"]}, 2]
(* {Quantity[5923/20, "Kelvins"]} *)

To get the plots to overlay correctly, convert back to Celsius.

Show[Histogram[data, Automatic, "PDF"], 
 ListLinePlot[
  Transpose[
   MapAt[UnitConvert[MovingAverage[#, 2], "DegreesCelcius"] &, 
    HistogramList[data, Automatic, "PDF"], 1]], PlotMarkers -> All]]

This looks like a bug to me, or at least unexpected, undocumented (as far as I can tell) and can result in unexpected errors. You should report it to Wolfram Support.

Thank you, Abrita. Seeing the documentation is what I badly needed !

-Marv

Reminder that the statistics group starts Monday! Author @David Withoff will join us as we kickoff our latest Daily Study Group. A pre-release version of the interactive course framework will be shared with participants. Sign up here.