FindMinimum Variable Independence Puzzle

Questions
,
1

Using an example from the NMinimize documentation:

f[x_, y_] =
E^Sin[50 x] + Sin[60 E^y] + Sin[70 Sin[x]] + Sin[Sin[80 y]] -
Sin[10 (x + y)] + 1/4 (x^2 + y^2);

I use FindMinimum from two different starting points:

In[7]:= FindMinimum[f[x, y], {{x, 1}, {y, 1}}]

Out[7]= {-0.342853, {x → 0.953282, y → 0.291998}}

In[8]:= FindMinimum[f[x, y], {{x, -1}, {y, -1}}]

Out[8]= {-1.40759, {x → -1.66754, y → 0.67725}}

The I try to do both starting points at once:

In[9]:= FindMinimum[
f[x1, y1] + f[x2, y2], {{x1, 1}, {y1, 1}, {x2, -1}, {y2, -1}}]

Out[9]= {-0.674649, {x1 → 0.994175, y1 → 0.993418, x2 → -1.00898,
y2 → -0.957112}}

and get a different answer. It seems to me the minimization of f[x1,y1] should be
independent of the minimization of f[x2,y2]

2

FindMinimum searches for a local minimum, so the answer will depend on the starting point. The particular function f[x,y] is very variable with what appears to be many local minima. Here just plotting it between -0.2 and 0.2:

f[x_, y_] := 
  E^Sin[50 x] + Sin[60 E^y] + Sin[70 Sin[x]] + Sin[Sin[80 y]] - 
   Sin[10 (x + y)] + 1/4 (x^2 + y^2);

ContourPlot[f[x, y], {x, -.2, .2}, {y, -.2, .2}]

and

Plot3D[f[x, y], {x, -.2, .2}, {y, -.2, .2}]

And one can get a different impression of how wobbly it is via (now plotting between -2 and 2):

Manipulate[Plot[f[x, y], {x, -2, 2}], {y, -2, 2}]

Also your In[8] and In[9] cases both give the following warning messages:

FindMinimum::lstol: The line search decreased the step size to within
the tolerance specified by AccuracyGoal and PrecisionGoal but was
unable to find a sufficient decrease in the function. You may need
more than MachinePrecision digits of working precision to meet these
tolerances.

But your question is really about why the order matters of the variable iterators in the FindMinimum function call. Perhaps the starting direction for the search for a local minimum depends on that order and therefore differing local minima are achieved when different initial search directions are chosen. I imagine that it depends on what the choice of Method is being used.

Suggestion: make use of the formatting tools for your code. Copying and pasting the code into the text causes characters to be deleted (or perhaps you left them out). And it makes it easier to copy your examples from the forum into Mathematica. It also makes it more readable.

3

On reflection, I think what’s coupling the different variables together is the backtracking in the line search. I tried a similar experiment with the COIN-OR solver IPOPT

and got the independence I was expecting.

callIpOpt[f[x, y], {}, {{x, -2, 1, 2}, {y, -2, 1, 2}}]

{-0.618508, {x -> 0.994175, y -> 0.993418}, "cons values" -> {}, 
 "var lb \[Lambda]s" -> {1.24674*10^-9, 1.24706*10^-9}, 
 "var ub \[Lambda]s" -> {3.71136*10^-9, 3.70856*10^-9}, 
 "cons \[Lambda]s" -> {}, "Solve_Succeeded"}

callIpOpt[f[x, y], {}, {{x, -2, -1, 2}, {y, -2, -1, 2}}]

{-0.0561414, {x -> -1.00898, y -> -0.957112}, "cons values" -> {}, 
 "var lb \[Lambda]s" -> {2.5286*10^-9, 2.40285*10^-9}, 
 "var ub \[Lambda]s" -> {8.32809*10^-10, 8.47416*10^-10}, 
 "cons \[Lambda]s" -> {}, "Solve_Succeeded"}

callIpOpt[
 f[x1, y1] + 
  f[x2, y2], {}, {{x1, -2, 1, 2}, {y1, -2, 1, 2}, {x2, -2, -1, 
   2}, {y2, -2, -1, 2}}]

{-0.674649, {x1 -> 0.994175, y1 -> 0.993418, x2 -> -1.00898, 
  y2 -> -0.957112}, "cons values" -> {}, 
 "var lb \[Lambda]s" -> {1.24674*10^-9, 1.24706*10^-9, 3.76679*10^-9, 
   3.57946*10^-9}, 
 "var ub \[Lambda]s" -> {3.71136*10^-9, 3.70856*10^-9, 1.24061*10^-9, 
   1.26237*10^-9}, "cons \[Lambda]s" -> {}, "Solve_Succeeded"}

Thanks for suggesting the formatting tool. I hadn’t realized cutting and pasting dropped things, like underscores.