Message Boards Message Boards


How to formulate minimization problems with vector solution variables?

Posted 5 months ago
3 Replies
2 Total Likes

Solve a linear system m.x=b with WL-Function Minimize:

In[1]:= m = {{10, 7, 5}, {9, 7, 4}, {1, 4, 10}};
b = {3, 5, 3};

MatrixForm[BoxForm`e$]]]\) . {x1, x2, x3} - b]}, {x1, x2, x3}]

Out[3]= {0, {x1 -> -(109/83), x2 -> 232/83, x3 -> -(57/83)}}

Works as expected.

But what does the error message mean when I represent the variable x as a vector? How can I formulate equations with vector solution variables?

In[4]:= Minimize[{Norm[\!\(\*
MatrixForm[BoxForm`e$]]]\) . x - b]}, 
 x \[Element] Vectors[3, Rationals]]

During evaluation of In[4]:= Minimize::objfs: The objective function {Sqrt[Abs[-5+10 Indexed[<<2>>]+7 Indexed[<<2>>]+5 Indexed[<<2>>]]^2+2 Abs[-3+Times[<<2>>]+Times[<<2>>]+Times[<<2>>]]^2],Sqrt[Abs[-5+9 Indexed[<<2>>]+7 Indexed[<<2>>]+4 Indexed[<<2>>]]^2+2 Abs[-3+Times[<<2>>]+Times[<<2>>]+Times[<<2>>]]^2],Sqrt[Abs[-5+Subscript[x, <<1>>]+4 Indexed[<<2>>]+10 Indexed[<<2>>]]^2+2 Abs[-3+Indexed[<<2>>]+Times[<<2>>]+Times[<<2>>]]^2]} should be scalar-valued.

Out[4]= Minimize[{Sqrt[
  Abs[-5 + {{10, 7, 5}, {9, 7, 4}, {1, 4, 10}} . x]^2 + 
   2 Abs[-3 + {{10, 7, 5}, {9, 7, 4}, {1, 4, 10}} . x]^2]}, 
 x \[Element] Vectors[3, Rationals]]
POSTED BY: Lars Ulke-Winter
3 Replies

Also, as another note: Since you are just doing least squares with a square, nonsingular matrix, you could just do

x = Inverse[m].b;

or more generally (for a non-square matrix):

x = PseudoInverse[m].b;

to get the least squares solution. But this may have just been an oversimplified example here and may not be applicable to you.

POSTED BY: David Trimas

I am not sure why, but I've had to sometimes use Inactive[] with my objective functions when using ConvexOptimization[] and I suspected that may also be the case here. I tried:

Minimize[Norm[Inactive[Plus][m.x,-b]],x \[Element]Vectors[3,Reals]] 

And it appears to work and produces the same solution as the first example. However, I had to change the solution space to Reals, it didn't work with rationals. I have a suspicion this doesn't matter since you can always find a rational number arbitrarily close to a real number. If all the components of the objective function's gradient at the minimizer are finite, a rational close to the minimizer will also have an objective function value close to that of the minimizer. but I could be very wrong about this as I don't know really anything about optimization over rationals.

POSTED BY: David Trimas

This is very interesting - thank you for posting this hint!

The problem seems to be that at some point expressions like

-5 + {{10, 7, 5}, {9, 7, 4}, {1, 4, 10}} . x

are to be evaluated - and this cannot work. Obviously by using Inactive[Plus] this can be prevented.

But this implies that the new function Threaded[] can help as well, like so:

Minimize[Norm[Threaded[m . x] - b], x \[Element] Vectors[3, Reals]]
POSTED BY: Henrik Schachner
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
or Discard

Group Abstract Group Abstract