Group Abstract Group Abstract

Message Boards Message Boards

Test the significance of the result from NonlinearModelFit?

Posted 6 years ago

I have some data, and I have done a NonlinearModelFit on it, actually fitting it to a sine curve. I can get the "RSquared" and "AdjustedRSquared", e.g. with nlm["AdjustedRSquared"] where nlm is the output of the NonlinearModelFit. I now want to test the significance of the result. I would like to end up with a single number p, so that I could say, "the probability of getting such a fit by chance is p".

NonlinearModelFit has properties like "ParameterPValues" and "ParameterTStatistics". However, I have looked in the StatisticalModelAnalysis tutorial, and there is no real explanation of how they might be used or generally how to do significance testing.

Does NonlinearModelFit have built in ways to get significance (probability of fit being due to chance)? Or is there a good tutorial on using the output of Mathematica's NonlinearModelFit to do significance testing?

POSTED BY: Marc Widdowson
5 Replies
Posted 6 years ago

Hope it was helpful.

There is a different interpretation between the $R^2$ calculated using LinearModelFit and NonlinearModel but not because of something intrinsically different between linear and nonlinear models but rather that Mathematica chooses to use two different formulas for $R^2$ for the two procedures.

LinearModelFit uses $1-{{SS_{res}}\over{SS_{corrected Total}}}$ and NonlinearModelFit uses $1-{{SS_{res}}\over{SS_{uncorrected Total}}}$. One can see this by fitting the same linear model in LinearModelFit and NonlinearModelFit.

POSTED BY: Jim Baldwin
Posted 6 years ago
POSTED BY: Jim Baldwin
Posted 6 years ago

Thank you very much. This is very helpful. The data I am working with is the percentage of "anocracies" (between democracy and autocracy) in the Polity IV database of regime types from 1800 to 2017, which you would not expect to be sinusoidal. It looks like it would probably fit a saw tooth wave as well as or better than a sine wave. However, a sine wave is the solution to a simple dynamic model (second derivative proportional to negative of current value), and so its presence provides a good starting point for developing a theory of what is going on.

The issue about R2 having a different interpretation is in the Mathematica Tutorial on Statistical Model Analysis. It says, "The coefficient of determination does not have the same interpretation as the percentage of explained variation in nonlinear models as it does in linear models because the sum of squares for the model and for the residuals do not necessarily sum to the total sum of squares." I'm not sure why this is, but I suppose it's to do with the non-linearity. It would be nice if they said how we might interpret it, but they don't...perhaps because it depends entirely on the situation.

POSTED BY: Marc Widdowson
Posted 6 years ago
POSTED BY: Marc Widdowson
Posted 6 years ago
POSTED BY: Jim Baldwin
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard