Message Boards Message Boards

Investigation - power outages in the UK

Posted 5 years ago
Attachments:
POSTED BY: Henry Jaspars
7 Replies

enter image description here - Congratulations! This post is now featured in our Staff Pick column as distinguished by a badge on your profile of a Featured Contributor! Thank you, keep it coming, and consider contributing your work to the The Notebook Archive!

POSTED BY: EDITORIAL BOARD
Posted 5 years ago

Good question - this is from a statistical hypothesis test (here, a one-tailed z-test, with p-value).

Assume the frequencies, F, are similar to the distribution NormalDistribution[50.0012, 0.0584802]. Then we have two hypotheses:

H0: This was caused by a statistical misfortune (the null hypothesis)

H1: This was caused by a process problem (the alternative hypothesis)

We will test this at the 99.99% significance level (so for us to be certain H1 was the case, the probability this would occur if the process was in statistical control must be below ? = 0.0001). The probability this was an accident, assuming the process was in control, was p = Prob[F < 49.5] = 5.1191*10^-18 < 0.0001 = ?. So we have sufficient evidence, at the 99.99% significance level, to reject H_0. Therefore, we can be almost (but never completely; such is the way of statistics) certain that this could not be a statistical misfortune. So the process was not in control at the time of the blackouts.

POSTED BY: Henry Jaspars

Thank you for your detailed answer. Some parts still not sufficiently explained, for example you are using a variable called newfreqs when you are comparing the distribution of the frequencies with Normal distribution instead of the usual variable frequencies but you didn't show what this newfreqs is. I am asking about that because the frequencies are not normally distributed

h = DistributionFitTest[Last /@ frequencies, Automatic, "HypothesisTestData"]
h["TestDataTable", All]

Long story short, I think it is too early to judge that the blackout is not by chance specially that it happened only twice in 8 years, so using the word systematic is harsh. You still can be write but it needs more investigation I think.

POSTED BY: Ahmed Elbanna
Posted 5 years ago

With regards to the variable names, I have just amended that. Thank you for pointing that out!

Although I agree that a normal distribution is not perfect here, observe how far the 49.5 Hz reading lies from the mean:

In[1]: (Mean[Last/@frequencies] - 49.5)/StandardDeviation[Last/@frequencies]

Out[1]: 8.57123

Regardless of choice of distribution, 8.57 standard deviations from the mean is a remarkable outlier. Which distribution would you suggest as opposed to the normal (I considered Student's T, but this was not successful)?

POSTED BY: Henry Jaspars
POSTED BY: Kapio Letto
Posted 5 years ago

I had already tried this, but was not pleased with the results.

In[1]: dist = FindDistrubution[Last/@frequencies]

Out[1]: StudentTDistribution[50.0054, 0.0739269, 8.64764]

In[2]: Show[Histogram[Last /@ frequencies, Automatic, "ProbabilityDensity"], 
 Plot[PDF[dist, x], {x, 49.7, 50.3}, 
  PlotStyle -> Thick]]

which gave:

graph

which doesn't seem like a particularly good fit. Others from the best solutions do not seem particularly accurate either.

POSTED BY: Henry Jaspars
POSTED BY: Ahmed Elbanna
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract