Message Boards Message Boards

0
|
2383 Views
|
2 Replies
|
2 Total Likes
View groups...
Share
Share this post:

FindMinimum Variable Independence Puzzle

Using an example from the NMinimize documentation:

f[x, y] = E^Sin[50 x] + Sin[60 E^y] + Sin[70 Sin[x]] + Sin[Sin[80 y]] - Sin[10 (x + y)] + 1/4 (x^2 + y^2);

I use FindMinimum from two different starting points:

In[7]:= FindMinimum[f[x, y], {{x, 1}, {y, 1}}]

Out[7]= {-0.342853, {x -> 0.953282, y -> 0.291998}}

In[8]:= FindMinimum[f[x, y], {{x, -1}, {y, -1}}]

Out[8]= {-1.40759, {x -> -1.66754, y -> 0.67725}}

The I try to do both starting points at once:

In[9]:= FindMinimum[ f[x1, y1] + f[x2, y2], {{x1, 1}, {y1, 1}, {x2, -1}, {y2, -1}}]

Out[9]= {-0.674649, {x1 -> 0.994175, y1 -> 0.993418, x2 -> -1.00898, y2 -> -0.957112}}

and get a different answer. It seems to me the minimization of f[x1,y1] should be independent of the minimization of f[x2,y2]

POSTED BY: Frank Kampas
2 Replies

FindMinimum searches for a local minimum, so the answer will depend on the starting point. The particular function f[x,y] is very variable with what appears to be many local minima. Here just plotting it between -0.2 and 0.2:

f[x_, y_] := 
  E^Sin[50 x] + Sin[60 E^y] + Sin[70 Sin[x]] + Sin[Sin[80 y]] - 
   Sin[10 (x + y)] + 1/4 (x^2 + y^2);

ContourPlot[f[x, y], {x, -.2, .2}, {y, -.2, .2}]

and

Plot3D[f[x, y], {x, -.2, .2}, {y, -.2, .2}]

And one can get a different impression of how wobbly it is via (now plotting between -2 and 2):

Manipulate[Plot[f[x, y], {x, -2, 2}], {y, -2, 2}]

Also your In[8] and In[9] cases both give the following warning messages:

FindMinimum::lstol: The line search decreased the step size to within the tolerance specified by AccuracyGoal and PrecisionGoal but was unable to find a sufficient decrease in the function. You may need more than MachinePrecision digits of working precision to meet these tolerances.

But your question is really about why the order matters of the variable iterators in the FindMinimum function call. Perhaps the starting direction for the search for a local minimum depends on that order and therefore differing local minima are achieved when different initial search directions are chosen. I imagine that it depends on what the choice of Method is being used.

Suggestion: make use of the formatting tools for your code. Copying and pasting the code into the text causes characters to be deleted (or perhaps you left them out). And it makes it easier to copy your examples from the forum into Mathematica. It also makes it more readable.

POSTED BY: David Reiss

On reflection, I think what's coupling the different variables together is the backtracking in the line search. I tried a similar experiment with the COIN-OR solver IPOPT

https://projects.coin-or.org/Ipopt

and got the independence I was expecting.

callIpOpt[f[x, y], {}, {{x, -2, 1, 2}, {y, -2, 1, 2}}]

{-0.618508, {x -> 0.994175, y -> 0.993418}, "cons values" -> {}, 
 "var lb \[Lambda]s" -> {1.24674*10^-9, 1.24706*10^-9}, 
 "var ub \[Lambda]s" -> {3.71136*10^-9, 3.70856*10^-9}, 
 "cons \[Lambda]s" -> {}, "Solve_Succeeded"}

callIpOpt[f[x, y], {}, {{x, -2, -1, 2}, {y, -2, -1, 2}}]

{-0.0561414, {x -> -1.00898, y -> -0.957112}, "cons values" -> {}, 
 "var lb \[Lambda]s" -> {2.5286*10^-9, 2.40285*10^-9}, 
 "var ub \[Lambda]s" -> {8.32809*10^-10, 8.47416*10^-10}, 
 "cons \[Lambda]s" -> {}, "Solve_Succeeded"}

callIpOpt[
 f[x1, y1] + 
  f[x2, y2], {}, {{x1, -2, 1, 2}, {y1, -2, 1, 2}, {x2, -2, -1, 
   2}, {y2, -2, -1, 2}}]

{-0.674649, {x1 -> 0.994175, y1 -> 0.993418, x2 -> -1.00898, 
  y2 -> -0.957112}, "cons values" -> {}, 
 "var lb \[Lambda]s" -> {1.24674*10^-9, 1.24706*10^-9, 3.76679*10^-9, 
   3.57946*10^-9}, 
 "var ub \[Lambda]s" -> {3.71136*10^-9, 3.70856*10^-9, 1.24061*10^-9, 
   1.26237*10^-9}, "cons \[Lambda]s" -> {}, "Solve_Succeeded"}

Thanks for suggesting the formatting tool. I hadn't realized cutting and pasting dropped things, like underscores.

POSTED BY: Frank Kampas
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract