You can use the Trace[] function to see intermediate steps in the evaluation of a Mathematica expression. I often find it more instructive to apply trace to individual parts of complicated expressions. This helps to simplify interpreting the results. Depending on the complexity of the calculation, intermediate results may not be as obvious as those displayed by Wolfram Alpha.

For example, the following code generates one term of a derivative of your expression for U. You can see how Mathematica evaluates the expression from inside out, first substituting for k, then evaluating B[k], and then its derivative, etc.

Trace[D[B[k], r]]

Also, I forgot to include a link to the on-line version of the introductory Wolfram Language book mentioned in my previous answer. http://www.wolfram.com/language/elementary-introduction/?source=nav

Again, I believe that most of your problems are simple syntax errors. For example, Mathematica does not use square brackets for grouping, but for function arguments, etc. Use (nested) parentheses instead. For details see the following link.

http://reference.wolfram.com/language/howto/UseBracketsAndBracesCorrectly.html

Also, commas are only used to separate expressions in lists, or function arguments. The comma at the end of the "U=" line is causing a problem. Semicolons are used to separate expressions on the same line, and to supress output. The D[f,x] function differentiates f with respect to its second argument, etc.

You might save yourself some time by scanning through something like Stephen Wolfram's recent introductory book on the Mathematica language, and/or the tutorials in the Mathematica Documentation Center.

I fixed some of the errors, and this may be closer to what you want.

Clear[r, R, y, Y, k, A, B, U, Ur, Uy]
k = ((4*R*r)/((y - Y)^2 + (r + R)^2))^1/2;
A[k_] := EllipticK[k];
B[k_] := EllipticE[k];
U = (R*r)^1/2/(2*Pi*k) ((2 - k^2)*A[k] - 2*B[k]); 
Ur = (1/r) D[U, y]
Uy = (-1/r) D[U, r]