0
|
7750 Views
|
7 Replies
|
7 Total Likes
View groups...
Share
GROUPS:

# MLE: NMaximize vs FindDistributionParameters

Posted 10 years ago
 2 simple ways to perform (simple) Maximum Likelihood Estimation in Mathematica: (1) Construct the Log-Likelihood function and maximize it: (2) Define the Probability Distribution and find parameters: They are not that much different, in the sense that they yield close values for the Log-Likelihood function (shown in the plot below), but still, the difference is noticeable in the resulting values of the parameters and I am curious which should I trust more? Attachments:
7 Replies
Sort By:
Posted 10 years ago
 Thanks, Marco. Usually the Hessian (or an approximation of it) is created during the iterative process for finding maximum likelihood estimates so I would think that it shouldn't be too much trouble to make that available (or its inverse when it exists) as an option for FindDistributionParameters.If Wolfram is packaging these high level functions to be of value other than being a good way to obtain starting values for a more detailed analysis, I would consider that providing these estimated standard errors is essential.Jim
Posted 10 years ago
 Maybe it's part of the documentation and I just don't see it but it appears for FindDistributionParameters, there is no option to obtain estimates of the standard errors and covariances of the parameter estimators. A Statistics golden rule is "An estimate without an associated estimate of precision is at best of unknown value."Bruce Miller: Where do I send such suggestions?
Posted 10 years ago
Posted 10 years ago
 Jim, Suggestions can be sent via http://www.wolfram.com/support/contact/email/?topic=Technical A simple code example with actual output and preferred output would help clarify the issue.
Posted 10 years ago
 I continue to make more than my fair share of programming mistakes (especially with Mathematica). The only ways I know how to minimize such occurrences are to (1) perform lots of checks (Do the probability functions sum/integrate to 1 for arbitrary parameters?), and (2) (probably the best technique) Grab anybody and explain your problem with the code in front of both of you. Then either that person will catch the obvious (upper/lowercase, spelling, etc.) or while you're explaining you'll say "Oh, nevermind." because the issue will become obvious.
Posted 10 years ago
 The two solutions are too far apart to be considered similar. The issue is that a factor of 1/2 is missing from the pdf definition. To match the likelihood the pdf needs to be pdf[x_] := Sqrt[(1 - \[Rho])/\[Rho]] Exp[(inN[x]^2 - ((Sqrt[1 - \[Rho]] inN[x] - inN[PD])/ Sqrt[\[Rho]])^2)/2]; Now the result will be {PD -> 0.013438593418697627473, \[Rho] -> 0.10995316859509961784} which matches {137.731, {PD -> 0.0134386, \[Rho] -> 0.109953}} 
Posted 10 years ago
 That's embarrassing..Thank you, Jim!