0
|
2418 Views
|
|
0 Total Likes
View groups...
Share
GROUPS:

# MLE using Truncated Data: NMaximize vs. EstimatedDistribution?

Posted 11 years ago
 I came across a curious phenomena in my work recently.  I am trying to fit a parametric distribution to some truncated data that also includes a weight for each observation using Mathematica.  Maximum likelihood obviously does this pretty neatly.  However, I used two different ways of doing it in Mathematica:d = WeightedData[rawdat[[All, 2]], rawdat[[All, 1]]]EstimatedDistribution[d, TruncatedDistribution[{0, 150000}, LogNormalDistribution[\[Mu], \[Sigma]]], {\[Mu], \[Sigma]}]]LogLikelihood[%[[2]], d]Let's say this gives me: mu = 9.86564 and sigma = 0.905846, and the log-likelihood has a value of -11.1108 at the solution.  However, this took 14 sec.Or I do this:NMaximize[{  (1/Total[rawdat[[All, 1]]]) *    Total[rawdat[[All, 1]].Log[PDF[TruncatedDistribution[{0, 150000}, LogNormalDistribution[\[Mu], \[Sigma]]], #] & /@ rawdat[[All, 2]]]],  \[Sigma] > 0}, {\[Mu],\[Sigma]}]]Now the answer is: mu = 9.86567 and sigma = 0.905853, though the log-likelihood has the same value at the solution as before.  Most importantly, this took only 2 sec.What gives?  Why are the solutions slightly different and why is EstimatedDistribution[] so much slower?  Is it that the WeightedData[] function is harder for Mathematica to deal with?  Does anyone here know?MarkusPS: This was a sub-sample (n = 250) of the full data that I am using.  With the full data, the difference in calculation time is 5+ minutes vs. 50 seconds.