I want to use my softmax as below not SoftmaxLayer[]. I have tried many times. But it still can’t work.
Below is my code now. How to rewrite it?
ElementwiseLayer[Exp[Normalize[#] &], Total]
ElementwiseLayer applies the function to each element of the inputlist. so it cannot calculate the mean.
You have to use FunctionLayer
In[39]:= soft = FunctionLayer[Exp[#]/Total[Exp[#]] &]
list = N@{1, 2, 3, 4, 5};
Exp[list]/Total[Exp[list]]
soft[list]
Out[39]= FunctionLayer[ <> ]
Out[41]= {0.0116562, 0.0316849, 0.0861285, 0.234122, 0.636409}
Out[42]= {0.0116562, 0.0316849, 0.0861285, 0.234122, 0.636409}
Thank you.