and the new notation : (x |-> x^2)[3]

Yes, I learnt that from you. Thanks.

How to tell if there is side effect?

There are two possible questions here. I'll answer both. If you're writing a function, and you want to make sure there are no side effects, then you must make sure that your function references only those variables that are passed in as arguments. If your function reference any other variable, then you have a side effect. This can be tricky if your function references another function, because that function might reference other functions. So, for example, if your function uses RandomInteger (or any of the random functions), then your function has a side effect, because RandomInteger depends on a special system variable. You could "fix" this by using SeedRandom, but then your function won't be random any more. In this case, you just accept the side effect. There might be other subtle ways to create side effects, but referencing non-argument variables is the main way.

The second question is how to tell whether a function that you didn't write but want to use has a side effect. Well, you'd have to inspect the definition of the function and understand it (and the functions that it references) well enough to determine whether side effects exist. And this is the problem with side effects. You may not be able to inspect the function defintion, or the side effect may be subtle. This is why side effects are so pernicious. Unexpected side effects are the cause of many, many software defects.

What is the difference between [MySquare3 and Function[countUsages += 1; #^2]]

Effectively there's no difference. The exact same computations occur. But the process for getting there was slightly different. To evaluate MySquare3[1], the evaluator needed to search the list of DownValues (among other things) for something matching that expression. If it finds something, it will do a replacement. In this case it would have found

HoldPattern[MySquare3[x_]] :> (countUsages += 1; x^2)

and so would have done the replacement to give

countUsages += 1; 1^2

which it would then proceed to evaluate further.

To evaluate Function[countUsages += 1; #^2][1], the evaluator didn't need to go looking for DownValues (or any other type of replacement). It just immediately slurped the argument into the function body to give

countUsages += 1; 1^2

It's the exact same result, just a slightly different path to get there.

what is MySquare3 /@ Range[3] ... why can you omit the [#]

This is just how this syntax is defined. The Map function, aka /@, applies its first argument to every element of the second argument at the first level. So, given a function f and an argument x, the word "apply" means "construct the expression f[x]".

You asked about MySquare3[#] /@ Range[3]. That's a syntactically valid expression--it just doesn't do what you want.

MySquare3[#] /@ Range[3]
(* {(#1^2)[1], (#1^2)[2], (#1^2)[3]} *)

To make this work with slots, you need to use Function:

MySquare3[#] & /@ Range[3]
(* {1, 4, 9} *)

This works, but the Function is superfluous. To evaluate MySquare3[#]&[1], we pull the argument into the body of the function to get MySquare3[1], and now we're right where we would have been had we just started with MySquare3 /@ Range[3].

In many cases, anonymous functions are pure functions, am I right?

I think it is probably true that anonymous functions tend to be pure. Anonymous functions tend to be small. Anonymous functions tend to be created ad hoc for some very specific purpose. These factors make it less likely that you will add side-effects. But I'm not seeing the relevance of the question.

I feel that whether it is a pure function is not important, the more important thing is we make functions to meet our requirements

Okay, we're drifting into philosophical waters. Having worked in the software industry for 25 years, I'm pretty confident in saying that a large proportion of bugs are due to side-effects. Are you sure that "meet our requirements" is straightforward? Let's say you've got a perfectly correct function doing some business computation. Then it's decided that every function must hook into the logging framework in order to get fine-grained observability, so you add a new argument to your function to support logging. Now someone maintaining the logging framework changes some schema expectations, and your function is now broken even though it's still correct per the functional requirements. This kind of thing happens all...the...time.

Look, I'm just pointing out that "pure" and "anonymous" are not equivalent. We can communicate more clearly if we use these terms accurately. I'm also tangentially implying that pure functions are generally less risky in the long run, but your mileage may vary. You're obviously free to use words however you want to, and you're obviously free to write your own code the way you want to. At the end of the day, I'm not invested at all in what decisions you make. Write your functions however you want.

I think this thread has conflated pure and anonymous.

No. Usually when people use "pure" they mean no side effects. They may include other characteristics as well. "Anonymous" means, well, anonymous. That is, an anonymous function is one that is created "on the fly" rather than being retrieved from some variable in the environment.

This expression is an anonymous function:

Function[x, x^2]

That expression in isolation is pretty useless. Once it's evaluated, it's effectively gone (you can reference execution history with %, so that's not strictly speaking true, but practically speaking it's gone). In the following, we're putting an anonymous function to use (presumably we don't expect to use it again later, so we didn't bother giving it a name):

Function[x, x^2] /@ Range[3]

If we did wan't to use it again, we might name it.

MySquare = Function[x, x^2]

Now we can use MySquare and know we're using the exact same function each time. Of course, another functionally equivalent way is this:

MySquare2[x_] := x^2

Side note, the explicit naming of the arguments has nothing to do with a function being anonymous. The following expressions are indeed anonymous, but it's not the absence of "x" that makes them so.

Function[#^2]
#^2&

The following function is not pure:

MySquare3[x_] := (countUsages += 1; x^2)

Incrementing countUsages is a side effect. The following example demonstrates this:

countUsages = 0;
MySquare3 /@ Range[3];
countUsages
(* 3*)

Without inspecting the definition of MySquare3, you wouldn't expect countUsages to change. You can also do this with an anonymous function:

countUsages = 0;
Function[countUsages += 1; #^2] /@ Range[3];
countUsages
(* 3 *)

I should add that, yes, I'm aware the Mathematica documentation often uses "pure" and "anonymous" interchangeably. I find this very strange. The way I justify it to myself is that an expression like MySquare2[x_] := x^2 is actually an assignment of DownValues via SetDelayed, and even though we colloquially say we're defining the MySquare2 "function", we're really defining replacement rules. On the other hand, Function[#^2] is "self-contained", that is, we don't need to look up replacement rules to figure out how to apply it. I think this "stand alone" quality is what the Mathematica documentation is trying to indicate with "pure". I'm not even really speculating, though, as much as I'm grasping at straws to justify this confusing jargon. I'm not even sure I think that is any different than "anonymous".

I don't know. How to know pure more deep?

Thanks for your reply. But why the Plot3D[Sin[x y], {x, 0, 3}, {y, 0, 3}, ColorFunction -> Function[{x, y, z}, Hue[z]]] doesn't use pure function? Sometimes it uses pure function, but some times it doesn't.

Yes. I know now.