Message Boards

WOLFRAM COMMUNITY

2371 Views

2 Replies

2 Total Likes

View groups...

Follow this post

Share this post:

GROUPS:

Wolfram Language Optimization Numerical Computation

FindMinimum Variable Independence Puzzle

Frank Kampas

Frank Kampas, Physicist at Large Consulting

Posted 11 years ago

Using an example from the NMinimize documentation: f[x, y] = E^Sin[50 x] + Sin[60 E^y] + Sin[70 Sin[x]] + Sin[Sin[80 y]] - Sin[10 (x + y)] + 1/4 (x^2 + y^2); I use FindMinimum from two different starting points: In[7]:= FindMinimum[f[x, y], {{x, 1}, {y, 1}}] Out[7]= {-0.342853, {x -> 0.953282, y -> 0.291998}} In[8]:= FindMinimum[f[x, y], {{x, -1}, {y, -1}}] Out[8]= {-1.40759, {x -> -1.66754, y -> 0.67725}} The I try to do both starting points at once: In[9]:= FindMinimum[ f[x1, y1] + f[x2, y2], {{x1, 1}, {y1, 1}, {x2, -1}, {y2, -1}}] Out[9]= {-0.674649, {x1 -> 0.994175, y1 -> 0.993418, x2 -> -1.00898, y2 -> -0.957112}} and get a different answer. It seems to me the minimization of f[x1,y1] should be independent of the minimization of f[x2,y2]

POSTED BY: Frank Kampas

2 Replies

Sort By:

Frank Kampas

Frank Kampas, Physicist at Large Consulting

Posted 11 years ago

On reflection, I think what's coupling the different variables together is the backtracking in the line search. I tried a similar experiment with the COIN-OR solver IPOPT https://projects.coin-or.org/Ipopt and got the independence I was expecting. callIpOpt[f[x, y], {}, {{x, -2, 1, 2}, {y, -2, 1, 2}}] {-0.618508, {x -> 0.994175, y -> 0.993418}, "cons values" -> {}, "var lb \[Lambda]s" -> {1.2467410^-9, 1.2470610^-9}, "var ub \[Lambda]s" -> {3.7113610^-9, 3.7085610^-9}, "cons \[Lambda]s" -> {}, "Solve_Succeeded"} callIpOpt[f[x, y], {}, {{x, -2, -1, 2}, {y, -2, -1, 2}}] {-0.0561414, {x -> -1.00898, y -> -0.957112}, "cons values" -> {}, "var lb \[Lambda]s" -> {2.528610^-9, 2.4028510^-9}, "var ub \[Lambda]s" -> {8.3280910^-10, 8.4741610^-10}, "cons \[Lambda]s" -> {}, "Solve_Succeeded"} callIpOpt[ f[x1, y1] + f[x2, y2], {}, {{x1, -2, 1, 2}, {y1, -2, 1, 2}, {x2, -2, -1, 2}, {y2, -2, -1, 2}}] {-0.674649, {x1 -> 0.994175, y1 -> 0.993418, x2 -> -1.00898, y2 -> -0.957112}, "cons values" -> {}, "var lb \[Lambda]s" -> {1.2467410^-9, 1.2470610^-9, 3.7667910^-9, 3.5794610^-9}, "var ub \[Lambda]s" -> {3.7113610^-9, 3.7085610^-9, 1.2406110^-9, 1.2623710^-9}, "cons \[Lambda]s" -> {}, "Solve_Succeeded"} Thanks for suggesting the formatting tool. I hadn't realized cutting and pasting dropped things, like underscores.

On reflection, I think what's coupling the different variables together is the backtracking in the line search. I tried a similar experiment with the COIN-OR solver IPOPT

https://projects.coin-or.org/Ipopt

and got the independence I was expecting.

callIpOpt[f[x, y], {}, {{x, -2, 1, 2}, {y, -2, 1, 2}}]

{-0.618508, {x -> 0.994175, y -> 0.993418}, "cons values" -> {}, 
 "var lb \[Lambda]s" -> {1.24674*10^-9, 1.24706*10^-9}, 
 "var ub \[Lambda]s" -> {3.71136*10^-9, 3.70856*10^-9}, 
 "cons \[Lambda]s" -> {}, "Solve_Succeeded"}

callIpOpt[f[x, y], {}, {{x, -2, -1, 2}, {y, -2, -1, 2}}]

{-0.0561414, {x -> -1.00898, y -> -0.957112}, "cons values" -> {}, 
 "var lb \[Lambda]s" -> {2.5286*10^-9, 2.40285*10^-9}, 
 "var ub \[Lambda]s" -> {8.32809*10^-10, 8.47416*10^-10}, 
 "cons \[Lambda]s" -> {}, "Solve_Succeeded"}

callIpOpt[
 f[x1, y1] + 
  f[x2, y2], {}, {{x1, -2, 1, 2}, {y1, -2, 1, 2}, {x2, -2, -1, 
   2}, {y2, -2, -1, 2}}]

{-0.674649, {x1 -> 0.994175, y1 -> 0.993418, x2 -> -1.00898, 
  y2 -> -0.957112}, "cons values" -> {}, 
 "var lb \[Lambda]s" -> {1.24674*10^-9, 1.24706*10^-9, 3.76679*10^-9, 
   3.57946*10^-9}, 
 "var ub \[Lambda]s" -> {3.71136*10^-9, 3.70856*10^-9, 1.24061*10^-9, 
   1.26237*10^-9}, "cons \[Lambda]s" -> {}, "Solve_Succeeded"}

Thanks for suggesting the formatting tool. I hadn't realized cutting and pasting dropped things, like underscores.

POSTED BY: Frank Kampas

David Reiss

David Reiss, Scientific Arts

Posted 11 years ago

FindMinimum searches for a local minimum, so the answer will depend on the starting point. The particular function f[x,y] is very variable with what appears to be many local minima. Here just plotting it between -0.2 and 0.2: f[x_, y_] := E^Sin[50 x] + Sin[60 E^y] + Sin[70 Sin[x]] + Sin[Sin[80 y]] - Sin[10 (x + y)] + 1/4 (x^2 + y^2); ContourPlot[f[x, y], {x, -.2, .2}, {y, -.2, .2}] and Plot3D[f[x, y], {x, -.2, .2}, {y, -.2, .2}] And one can get a different impression of how wobbly it is via (now plotting between -2 and 2): Manipulate[Plot[f[x, y], {x, -2, 2}], {y, -2, 2}] Also your In[8] and In[9] cases both give the following warning messages: FindMinimum::lstol: The line search decreased the step size to within the tolerance specified by AccuracyGoal and PrecisionGoal but was unable to find a sufficient decrease in the function. You may need more than MachinePrecision digits of working precision to meet these tolerances. But your question is really about why the order matters of the variable iterators in the FindMinimum function call. Perhaps the starting direction for the search for a local minimum depends on that order and therefore differing local minima are achieved when different initial search directions are chosen. I imagine that it depends on what the choice of Method is being used. Suggestion: make use of the formatting tools for your code. Copying and pasting the code into the text causes characters to be deleted (or perhaps you left them out). And it makes it easier to copy your examples from the forum into Mathematica. It also makes it more readable.

POSTED BY: David Reiss

Reply to this discussion

Reply Preview

Attachments

Remove Add a file to this post

Follow this discussion

or Discard

Group Abstract

Feedback