Hi Ettore,
For this question
2) How can I initialize the random matrix w so that I don't have to
define it as a global variable?
one way to do it in WL is to define a symbol that returns a function with the value of w
locally bound. e.g.
randomProject = With[{w = RandomReal[{-1, 1}, {100, numFeats}]}, Function[data, Tanh[w.data]]];
Each time the above definition is evaluated, it will return a function with a different random matrix. Multiple evaluations of the function returned will use the same random matrix.
randomProject[train[[1, 1]]] == randomProject[train[[1, 1]]]
(* True *)