Dear everyone: I have wrote a program to implement gradient descent algorithm for linear fitting model. The linear fitting model is very simple: y = w0 + w1*x. I have sample data X={0,1,2,3,4} and Y={1,3,7,13,21}. First of all, I write a function to calculate the residual sum of square.

RSS[x_List, y_List, w0_, w1_] /; Length[x] == Length[y] := Module[
   {len = Length[x]},
   Return[Sum[(y[[i]] - (w0 + w1 x[[i]]))^2, {i,1,len}]]
   ];

Then I test function RSS like this:

In[58]:= RSS[X, Y, w0, w1]
Out[58]= (1 - w0)^2 + (21 - w0 - 4 w1)^2 + (13 - w0 - 3 w1)^2 + (7 - 
   w0 - 2 w1)^2 + (3 - w0 - w1)^2

After that, I wrote gradient descent algorithm as below:

GradientDescend[X_List, Y_List, initialIntercept_: 0, initialSlop_: 0, u_: 0.05, tolerance_: 0.01] /; Length[X] == Length[Y] := Module[
   {wList = {}, w0, w1, gr, k = 1},
   gr := Simplify /@ Grad[RSS[X, Y, w0, w1], {w0, w1}];
   w0 = initialIntercept;
   w1 = initialSlop;
   While[Norm[gr] > tolerance,
    {w0, w1} = {w0, w1} - u gr;
    Print["Iteration ", k, "======>", {w0, w1}];
    wList = Append[wList, {w0, w1}];
    k++;
    ];
   Return[wList]
   ];

In this function, initial Intercept is the initial w0 and initial slop is the initial w1. u is the step size and tolerance is the threshold. I want to return a list which records all w0 and w1. However, when I test it, I found I cannot get the correct answer:

In[59]:= GradientDescend[X, Y]

During evaluation of In[59]:= Grad::nocoord: {0,0} is not a non-empty list of valid variables.

During evaluation of In[59]:= Grad::nocoord: {0,0} is not a non-empty list of valid variables.

Out[59]= {}

Even if I clear w0 and w1 firstly, the error still occurs. I want to know what is wrong in my program and how to control the symbol argument w0 and w1? Thank you!