I still can't run this when NoVariables = 400. This is a convex objective function, so thought the optimization should be quite straightforward.

This method seems to run quite fast in Matlab:

"The Polack-Ribiere flavour of conjugate gradients is used to compute search directions, and a line search using quadratic and cubic polynomial approximations and the Wolfe-Powell stopping criteria is used together with the slope ratio method for guessing initial step sizes."

I couldn't find a similar method in Mathematica, only Method -> "ConjugateGradient". But this still does not produce any answer at all for 400 variables, even if I set MaxIterations to 1 (computer seems to freeze even though I'm working on a moderate spec one).

Any help would be much appreciated! Thanks.

This

Clear["Global`*"];
(*SeedRandom[100];*)
Objective=Compile[{{X,_Real,2},{Y,_Real,1},{Thetas,_Real,1}},
Module[{NoExamples=Length@X,xt=X.Thetas},
(1./NoExamples)*(Total[Y*Log[1+Exp[-xt]]-(-1+Y)*Log[1+Exp[xt]]])],"RuntimeOptions"->"Speed"];
NoVariables=10;
NoExamples=5000;
allData=RandomReal[1,{NoExamples,NoVariables}];
X=Transpose[Prepend[Transpose[allData],Table[1.,{NoExamples}]]];
Dimensions@X  (*X is 200 by 101 matrix,with first column of only 1's*)
Thetas=Table[Symbol["\[Theta]"<>ToString@i],{i,0,NoVariables}];
Dimensions@Thetas (*Thetas are 101 size vector,with Subscript[\[Theta],0] representing the coefficient of the constant term*)
Y=RandomChoice[{0.,1.},NoExamples];
Dimensions@Y (*y is a vector of size 200,being either 0 or 1*)
Timing@FindMinimum[Objective[ X,Y,Thetas],Thetas,Method->"PrincipalAxis"]

needs 2.5 seconds (in V10 there is a strange CompiledFunction::cfta warning message, which is not there in V9)

Thanks Daniel. I still don't seem to be able to do an example with NoVariables = 400. Do you think this may be out of bounds for Mathematica?