The residual in the formula for the residual sum of squares (rss) needs to be of the form "observed minus predicted". You have

zcol - a*xcol + b*ycol + c

and I suspect you dropped the parentheses around

a*xcol + b*ycol + c

So you need to have

zcol - (a*xcol + b*ycol + c)

Depending on your objective you might consider the following modification. I recommend using the "root mean square error" as it is in the same units as the predicted variable and gives similar values no matter what the sample size is. I've also added in a linear model fit so that the Manipulate starts out at the smallest root mean square error (rmse). And finally using the RotationAction -> "Clip" is essential if you want the user to have a less dizzying time rotating the 3D figure.

t = Import[
  "http://www.doc.gold.ac.uk//~ma302jh//RegressionExample.xls"]; tp = 
 t[[1]]; t1 = Table[tp[[i]], {i, 2, Length[tp]}]; xcol = 
 Table[t1[[i]][[1]], {i, 1, Length[t1]}]; ycol = 
 Table[t1[[i]][[2]], {i, 1, Length[t1]}]; zcol = 
 Table[t1[[i]][[3]], {i, 1, Length[t1]}];

(* Regression *)
fit = LinearModelFit[t1, {x, y}, {x, y}];
(* Parameter estimates *)
{cBest, aBest, bBest} = fit["BestFitParameters"];
(* Root mean square error *)
fit["EstimatedVariance"]^0.5

Clear[DefineRSS]; 
DefineRSS[a_?NumericQ, b_?NumericQ, 
  c_?NumericQ] := (rmse = (Total[(zcol - (a*xcol + b*ycol + 
            c))^2]/(Length[zcol] - 3))^0.5; (a*x) + (b*y) + c);
 Manipulate[
 With[{plane = DefineRSS[a, b, c]}, 
  Show[Plot3D[plane, {x, -10, 1000}, {y, -10, 60}, 
    AxesLabel -> {X, Y, Z}, PlotLabel -> rmse, 
    PlotStyle -> 
     Directive[Yellow, Specularity[White, 20], Opacity[0.5]], 
    RotationAction -> "Clip"], 
   Graphics3D[{Red, PointSize[Medium], 
     Point[Transpose[{xcol, ycol, zcol}]], 
     Line[Transpose[{{xcol, ycol, zcol}, {xcol, ycol, 
         plane /. Thread[{x, y} -> {xcol, ycol}]}}, {2, 3, 1}]]}], 
   PlotRange -> {{0, 1000}, {0, 60}, {-100, 500}}]],
 {{a, aBest}, 0, 1, Appearance -> "Labeled"},
 {{b, bBest}, -1, 5, Appearance -> "Labeled"},
 {{c, cBest}, -60, 300, Appearance -> "Labeled"},
 TrackedSymbols :> {a, b, c}]