Message Boards Message Boards

Use K-Leave Out Cross Validation method for fitting?

Posted 7 years ago

How can I test my model fits well using K-Leave Out Cross Validation say K=2. That means take out first 2 data and fit the model for the rest 10 data to find parameters and use 2 data for finding error and do this 6 times for this case.

I also asked the question at mathematica.stackexchange.com here

Here is my data and model. I tried something but it does not work properly. Any suggestion? Thanks in advance.

    data={{1.*10^-10,2.58022},{7.,2.66034},{7.33333,3.18999},{7.66667,3.88375},{8.,4.13342},{8.33333,4.21529},{8.66667,4.22846},{9.,4.2579},{9.33333,4.28118},{9.66667,4.28596},{10.,4.29661},{11.,4.31207}};
 data = TakeDrop[data, {#, # + 1}] & /@ Range[1, 12, 2] 

parameters = {\[Gamma], \[Epsilon], k, c, p, is}; 

f[x_] := (E^-k  \[Gamma])/(1 + c/(1 + (x/is)^p) + E^-k) + \[Epsilon] 
Do[model[j] =   Sum[( f@(Last@data[[j]])[[i, 1]] - (Last@data[[j]])[[i, 2]])^2, {i, 1, 10}];
 fit[j] =   NMinimize[{model[j], \[Gamma] > 0 && c > 0 && p > 0 && 
      0 < is <= 11}, parameters] // Chop;
 Thread[{\[Gamma], \[Epsilon], k, c, p, is} =    parameters /. Last@fit[j]];
 err[j] =   Total@Abs[
    f@(First@data[[j]])[[All, 1]] - (First@data[[j]])[[All, 2]]];
 Print[err[j]], {j, 1, 6}]
Attachments:
POSTED BY: Okkes Dulgerci
Posted 7 years ago

Here how I did..

data = {{1.*10^-10, 2.58022}, {7., 2.66034}, {7.33333, 
    3.18999}, {7.66667, 3.88375}, {8., 4.13342}, {8.33333, 
    4.21529}, {8.66667, 4.22846}, {9., 4.2579}, {9.33333, 
    4.28118}, {9.66667, 4.28596}, {10., 4.29661}, {11., 4.31207}};
data = TakeDrop[data, {#, # + 1}] & /@ Range[1, 12, 2];
f[j_, x_] := (E^-k[j]   \[Gamma][j])/(
   1 + c[j]/(1 + (x/is[j])^p[j]) + E^-k[j]) + \[Epsilon][j];
crossVal[j0_] :=  Module[{j = j0}, 
  model[j] =  Sum[( f[j, (Last@data[[j]])[[i, 1]]] - (Last@data[[j]])[[i, 
       2]])^2, {i, 1, 6}]; 
  fit[j] = NMinimize[{model[j], \[Gamma][j] > 0 && \[Epsilon][j] > 0 &&
        p[j] > 0 && 0 < is[j] <= 11}, {\[Gamma][j], \[Epsilon][j], 
      k[j], c[j], p[j], is[j]}] // Chop;
  Thread[{\[Gamma][j], \[Epsilon][j], k[j], c[j], p[j], 
     is[j]} = {\[Gamma][j], \[Epsilon][j], k[j], c[j], p[j], 
      is[j]} /. Last@fit[j]];
  err[j] = Total@Abs[  f[j, (First@data[[j]])[[All, 1]]] - (First@data[[j]])[[All, 2]]];
  Return[err[j]]]
Table[crossVal[i], {i, 6}]
POSTED BY: Okkes Dulgerci
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract