As an update here are some links reporting similar behaviour sometimes by design, sometimes clearly buggy:

http://stackoverflow.com/questions/6867892/reducing-memory-usage-in-an-extended-mathematica-sessionhttp://groups.google.com/group/comp.soft-sys.math.mathematica/msg/f95a9588b9803f54http://forums.wolfram.com/mathgroup/archive/2011/Aug/msg00012.htmlSummarizing here are some typical examples where Mathematica does not clear Module generated temporary variables:

1.

Module[{x}, x /. NDSolve[x'[t] == 1 && x[0] == 0, x, {t, 0, 1}][[1]]]

Result: x$NNN is not removed.

Reason: ¿ NDSolve stores some expressions referencing x$NNN in the system cache ?

Solution: Clear system cache. (Presumably this will happen on its own as the kernel session progresses.)

2.

Module[{y, z, a, b}, a = y + z; b = 2*a];

Result: y$NNN and z$NNN are not removed from global context.

Reason: y$NNN and z$NNN appear in the module output and are thus referenced by Out.

Solution: Clear Out, or more systematically limit the History Length through $HistoryLength

3.

Module[{x}, f = x]

Result: x$NNN is not removed from global context.

Reason: Definition of f is referencing x$NNN

Solution: Clear f.

4.

a[b_] := Module[{x}, x := 1; x /; b]

a[True]

Result:a x$NNN is permanently placed in the global context every time a is called.

Reason: Unknown. This appears to be a bug. (Clear and ClearSystemCache[] do not remove the x$NNN.)

5.

Module[{a, b},

a[i_] := b[i - 1];

b[i_] := a[i - 1];

a[0] = 0; b[0] = 0; a[10]]

Result: a$NNN and b$NNN remain in global context.

Reason: Unknown. Bug?

It would be very helpful if someone could explain the behaviour of 4. and 5. (and how to avoid it).