Simply remove the colon ":" from the definition of g
. With the colon, you will call the gradient when the variables w0, w1
are numerical, instead of symbolic, whence the errors. Here is the code with a couple of minor simplifications:
RSS[x_List, y_List, w0_, w1_] /;
Length[x] == Length[y] := (y - (w0 + w1 x)).(y - (w0 + w1 x)),
GradientDescend[X_List, Y_List, initialIntercept_: 0, initialSlop_: 0,
u_: 0.05, tolerance_: 0.01] /; Length[X] == Length[Y] :=
Module[{wList = {}, w0, w1, gr, k = 1},
gr = Simplify /@ Grad[RSS[X, Y, w0, w1], {w0, w1}];
w0 = initialIntercept;
w1 = initialSlop;
While[Norm[gr] > tolerance,
{w0, w1} = {w0, w1} - u gr;
Print["Iteration ", k, "======>", {w0, w1}];
wList = Append[wList, {w0, w1}];
k++;];
wList];