Group Abstract Group Abstract

Message Boards Message Boards

[WSG23] Daily Study Group: Introduction to Statistics

A Wolfram U Daily Study Group on "Introduction to Statistics" begins on April 3, 2023.

Join a cohort of fellow statistics enthusiasts to learn about collecting, describing, analyzing and interpreting data and trends in science, industry and society. Learn about techniques for data visualization and descriptive statistics, methods for calculating confidence intervals and tools for hypothesis testing from video lessons created by veteran instructor and developer David Withoff. Participate in live Q&A and review your understanding through interactive in-session polls. Complete quizzes at the end of the study group to get your certificate of program completion.

April 3-14, 2023, 11am-12pm CT (4-5pm GMT)

REGISTER HERE

enter image description here

Please feel free to use this thread to collaborate and share ideas, additional resources and questions with the instructors as well as with other members of the study group.
POSTED BY: Abrita Chakravarty
47 Replies
Sort By:

Hi;

When going through our class notes and in particular, lesson 23 "Confidence Intervals for Proportions" on page 3 about halfway down the page is a line of code for calculating Z* as follows:

SuperscriptBox["z", "*"]=InverseCDF[NormalDistribution[0,1],0.975]

Where is the 0.975 number coming from in this line of code and where is the 95% Confidence Level that we are trying to achieve appear in the calculations that we are trying to achieve, as stated in the line of text immediately above the line of code? It is obliviously an important number because if it changes then z* changes and with it the rest of the values.

Thanks so much,

Mitchell Sandlin
POSTED BY: Mitchell Sandlin
Dave Withoff
Dave Withoff
Posted 2 years ago
POSTED BY: Dave Withoff

David -

RE: Resource for Conditional Probability Problems

Your remark during the Statistics DSG that the internet is a great source of problem solving help was right on target. The following link goes to some discussion and a collection of solved problems in conditional probability. Working through these problems helps develop the ability to understand what is being asked in conditional probability problems and what techniques can be used to solve them.

https://www.studocu.com/ph/document/central-luzon-state-university/differential-equation-l/prob3160ch4-math/33698873
POSTED BY: Joseph Smith

The Wolfram U course Introduction to Statistics is now available with updated content (based on the feedback we received from DSG attendees) and also features a Final Exam. We would like to encourage all our DSG attendees to take the exam and claim their Level 1 certification. Good luck!
POSTED BY: Abrita Chakravarty

I received an email today reminding me to complete the quizzes and exam for the statistics DSG. However, requesting an exam does not work. There is just an endless loading symbol.

POSTED BY: Joseph Smith
POSTED BY: Jamie Peterson
Bob Renninger
Bob Renninger
Posted 3 years ago

Hi Jamie,

I notice that the final exam is now active in the course framework. Since an official announcement has not been made, I am thinking that the exam is not quite ready. In any case, I would like to privately report an error that I found, without disclosing anything about the final to the group. Please let me know where I can send an email about that.

Regards,

Bob
POSTED BY: Bob Renninger
POSTED BY: Jamie Peterson
Bob Renninger
Bob Renninger
Posted 3 years ago

Thanks, I have reported the issues.
POSTED BY: Bob Renninger

Dear Jose and Arben,

I suspect that there are two errors in the Quiz 6.

Problem 8: There is no correct answer. Multiple choice D) can be an answer but only when it is modified as "Approximately normal and with a standard error of 80g."

Problem 10: There is no answer. Even if we change the order of the two samples in MeanDifferenceCI, still there is no correct multiple choice.

Please see the attached

Attachments:
Quiz 6 Errors.nb
POSTED BY: Hee-Young Shin
POSTED BY: Arben Kalziqi
José Dordá
José Dordá
Posted 3 years ago

Thank you Arben for your time.
POSTED BY: José Dordá

Of course! Thanks for helping us catch errors. Even with lots of review, things still sometimes get through...
POSTED BY: Arben Kalziqi
José Dordá
José Dordá
Posted 3 years ago

About quizes:

While going through the quizes, it seems that there is no correct option (answer) for Quiz 6, Question 10. I tried with Mathematica and also an specialized statistics software and none of the option correspond to the correct answer for the data supplied.
POSTED BY: José Dordá
POSTED BY: Arben Kalziqi
Tom Ogilvy
Tom Ogilvy
Posted 3 years ago

Quiz 15, Problem 4, What is the expected value of the mean from a sampling distribution of sample size n, drawn from a normally distributed population with standard deviation σ? Goin g through the possible answers provided, the answer that is shown as correct is dependent on the sample problem in the video for nb 44. The central limit theorem governs this situation in this problem and it says the expected value of the mean from a sampling distribution equals the mean (µ) of the population. what you offer is the numerical value of the µ from the specific situation in the sampling problem.

POSTED BY: Tom Ogilvy

Hi Tom,

Thanks for bringing this to our attention. We're rolling out a fix; you may have to clear your cookies/cache to see it, but the answer will be updated.
POSTED BY: Arben Kalziqi

Hi;

When I try to request my final exam from the course framework, nothing happens. I get a wait indicator that never produces a final exam.
POSTED BY: Mitchell Sandlin

Hi @Mitchell Sandlin, the exam has not yet been added to the course framework. We will send an email notification to the Study Group participants when this is available next week.
POSTED BY: Jamie Peterson

Hi;

I am attempting to weight temperature data to use in creating a Normal Distribution using the EstimateDistribution function - see attached notebook. However, it seems that all I am getting is a bunch of messages with the processing failing to create a Normal Distribution. Please tell me what I am doing incorrectly.

Thanks,

Mitch Sandlin

Attachments:
Weighting_Question.nb
POSTED BY: Mitchell Sandlin
Rohit Namjoshi
Rohit Namjoshi
Posted 3 years ago

Take a look at

PDF[NormalDistribution[], #] & /@ wdAllFlat

The temperature values are far from 0 which is the mean used by NormalDistribution[], so the probability is tiny.
POSTED BY: Rohit Namjoshi

Hi Rohit;

In implement your suggestion, I did not see much change - see attached notebook. However, there is a good possibility that I misunderstood your suggestion and implemented it incorrectly.

As a little background, I ran exactly the same calculations without weighting the data, and all the probability calculations, including the NormalDistribution[] function, looked reasonable. However, I wanted to run the same calculations with weighted data to take into account that the current temperatures may be a better approximation due to climate change, and therefore assign more weight to the current temperatures and less weight to the temperatures 20 years ago.

Thanks,

Mitch Sandlin

Attachments:
Weighting_Question.nb
POSTED BY: Mitchell Sandlin
Dave Withoff
Dave Withoff
Posted 3 years ago
POSTED BY: Dave Withoff
Attachments:
TTestQuestions.nb
POSTED BY: Joseph Smith
Dave Withoff
Dave Withoff
Posted 3 years ago
POSTED BY: Dave Withoff

David,

Thanks for your response. I will need to digest this carefully but I really appreciate your help.

Joe Smith

POSTED BY: Joseph Smith

Thanks again. Some additional experiments with data where the mean moves away from 1000 illustrate the interaction between PValue, the significance level, and the conclusion of the TTest with respect to rejection of the null hypothesis.

Attachments:
Example_pValue_S...nb
POSTED BY: Joseph Smith

In the lecture today on the Multiple Testing Problem, a website was mentioned of someone who made it his hobby to collect false positives, or "spurious correlations", e.g., a time series of people drowning in swimming pools over successive years that appears to be very similar to the time series of movies coming out starring Nicholas Cage. The website shows many more funny coincidences.

This is the website: https://www.tylervigen.com/spurious-correlations

There's a book as well by the same author: https://www.amazon.com/Spurious-Correlations-Tyler-Vigen/dp/0316339431
POSTED BY: Ruben Yvan Maarten Fossion
POSTED BY: Joseph Smith

Unfortunately the course framework is not ready for release yet. We are only sharing a beta version with our study group attendees. We can include the link in our reminder emails, so you can get to it, even if you miss the live session.

POSTED BY: Abrita Chakravarty

Thanks. Understood.

POSTED BY: Joseph Smith

Hi;

In using both functions NormalCI and MeanCI, I am getting some unexpected results - please see attached notebook.

Thanks,

Mitch Sandlin

Attachments:
NormalCI_Problem.nb
POSTED BY: Mitchell Sandlin
lara wag
lara wag
Posted 3 years ago

Hi Mitch,

I suspect it has something to do with the handling of temperatures and temperature differences, see the link in Abrita's post.

I'm always a bit cautious when it comes to dimensionally significant quantities. Especially with large data sets, I have the feeling that the calculations are faster with pure numbers. Therefore, just like you, I would have loaded the temperatures first and put them into a consistent format. However, then I would have dropped the units. You can do that for example like this:

wdAllFlat = QuantityMagnitude@Flatten[wdAll];

With dimensionless numbers you get the expected results (as you have already shown with your "copy&paste approach").

A comment on your second question: The variable wdAllFlatt still has units in your case - therefore you get a normal distribution with units (http://reference.wolfram.com/language/ref/QuantityDistribution.html). However, if you work with pure numbers, then you will get the result with pure numbers only - this makes plotting much easier, for example...

By the way, this is not a speech against the use of units in WL - on the contrary: I am a big fan of it; but not in every context.
POSTED BY: lara wag
lara wag
lara wag
Posted 3 years ago

In the notebook "11.NumericalSummariesOfData.nb" there are images of this kind: picture taken from notebook You animated these diagrams in the accompanying video and I wondered how you did it. Unfortunately, there are only pictures in the notebook (which makes sense, since you have explanations in the notebook that are "on the soundtrack" in the video). Even if it's a little off-topic, but: would you be so kind and share with us the code of this stunning animation?
POSTED BY: lara wag
Dave Withoff
Dave Withoff
Posted 3 years ago
POSTED BY: Dave Withoff
lara wag
lara wag
Posted 3 years ago

Dear Mr. Withoff,

Thank you very much for your answer.

You are absolutely right, the code does distract a lot from the actual topic. There is also a separate course on the topic of "animations" here on WolframU.

Still, thanks for sharing, I found your method of moving the five points or the single point via the join function (pts = ...) very enlightening. Also that you were able to assemble the final image with only "Epilog" was not something I would have expected.
POSTED BY: lara wag

Hi;

I am interested in finding the value of "x" in a probability when I know the probability in which I am interested. Intuitively one would assume that the Solve function should produce the answer by simply solving for x - see below. However, Solve, SolveValue or Reduce does not return the desired results. In some situations, I know the probability in which I am interested but I do not know the value of x that gives that probability without a lot of trial and error, so I hope someone could point me in the right direction.

Thanks, Mitch Sandlin

Solve[0.6 == Probability[x, x [Distributed] NormalDistribution[998, 202]], x]
POSTED BY: Mitchell Sandlin

Dear Mr. Sandlin:

I am NO expert on Wolfram or Probability but the following may help you.

Attachments:
Sandlin-solve-pr...nb
POSTED BY: Juan Ortiz Navarro

Hi Juan;

Thanks so much. It was actually the InverseCDF function that performed the correct calculation.

Mitch Sandlin
POSTED BY: Mitchell Sandlin

How to take the Quizzes and the Online Course Exam?
POSTED BY: Md Mohsin
POSTED BY: Abrita Chakravarty

Show does not work properly in Mathematica 13.2.x. The overlay shown in the downloaded notebook 8.UsingHistogramData.nb fails to show the overlay!

Attachments:
8.UsingHistogramData.nb
POSTED BY: Marvin Schaefer
Rohit Namjoshi
Rohit Namjoshi
Posted 3 years ago

Hi Marvin,

The reason is that Show takes PlotRange from its first argument (Histogram) and you will notice that the second argument (ListLinePlot) has a very different PlotRange. Why???

Because, unexpectedly, MovingAverage performs a unit conversion from Celcius to Kelvin.

MovingAverage[{Quantity[22, "DegreesCelsius"], Quantity[24, "DegreesCelsius"]}, 2]
(* {Quantity[5923/20, "Kelvins"]} *)

To get the plots to overlay correctly, convert back to Celsius.

Show[Histogram[data, Automatic, "PDF"], 
 ListLinePlot[
  Transpose[
   MapAt[UnitConvert[MovingAverage[#, 2], "DegreesCelcius"] &, 
    HistogramList[data, Automatic, "PDF"], 1]], PlotMarkers -> All]]

This looks like a bug to me, or at least unexpected, undocumented (as far as I can tell) and can result in unexpected errors. You should report it to Wolfram Support.
POSTED BY: Rohit Namjoshi

There was an update to how temperature units are handled in 13.2. This should provide an explanation: https://reference.wolfram.com/language/tutorial/TemperatureUnits.html
POSTED BY: Abrita Chakravarty
POSTED BY: Marvin Schaefer

Thank you very much for diagnosing the problem, Rohit! I got a response from support about the units on the x-axis not corresponding between the ListLinePlot and the graphic, and that was also not making sense to me at the time. Your response made it all very clear and, as you indicate, I do not think that this modification in implicit conversion was not documented in the prerelease documents I received for 3.2 or 3.3. It certainly is not compatible with the documentation and is possibly an easily corrected bug.

I truly appreciate your persisting in analysis of this conundrum! I’ll re-report it to Support.

-Marv Schaefer
POSTED BY: Marvin Schaefer

Reminder that the statistics group starts Monday! Author @David Withoff will join us as we kickoff our latest Daily Study Group. A pre-release version of the interactive course framework will be shared with participants. Sign up here.
POSTED BY: Jamie Peterson
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard