How to implement a normalization layer that does not exponentiate (like softmax)?

Questions
,
1

It should sum up all the elements from an input vector and just divide each one by that sum. Assume all elements are positive.

2

You mean something like this?

NetGraph[
 <|"norm" -> AggregationLayer[Total, 1], 
  "divide" -> ThreadingLayer[Divide, -1]|>,
 {NetPort["Input"] -> "norm", {NetPort["Input"], "norm"} -> "divide"}
 ]
3

Yes, exactly! Thanks for your patience while I get up to speed.

4

Just for people who might need to try this some day: I needed a ramp layer, because otherwise the NN would sometimes give negative numbers for probabilities that were close to 0 and that would mess up the normalization and all the other probabilities.