Group Abstract Group Abstract

Message Boards Message Boards

0
|
2.4K Views
|
3 Replies
|
1 Total Like
View groups...
Share
Share this post:

How to implement a normalization layer that does not exponentiate (like softmax)?

Posted 11 months ago

It should sum up all the elements from an input vector and just divide each one by that sum. Assume all elements are positive.

POSTED BY: Iuval Clejan
3 Replies
Posted 11 months ago

Just for people who might need to try this some day: I needed a ramp layer, because otherwise the NN would sometimes give negative numbers for probabilities that were close to 0 and that would mess up the normalization and all the other probabilities.

POSTED BY: Updating Name
Posted 11 months ago

Yes, exactly! Thanks for your patience while I get up to speed.

POSTED BY: Iuval Clejan

You mean something like this?

NetGraph[
 <|"norm" -> AggregationLayer[Total, 1], 
  "divide" -> ThreadingLayer[Divide, -1]|>,
 {NetPort["Input"] -> "norm", {NetPort["Input"], "norm"} -> "divide"}
 ]
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard