NN with neurons that are designed as a Sum of Weighted Inputs is no good at solving problems where the Output Targets are, in part, a Product or a Function of multiple inputs. Is there a NN architecture that can solve problems like Target = Input1 * Input 2 + Input3? What do these architectures look like? Do these Input Products of multiple Inputs need to be represented as additional Inputs like Input6 = Input1 * Input2? Or should permutations of Input Products be treated as additional Hidden Layers as a Sum of Weighted Products of Inputs? Then what about Target = Input1 * Input2 * Input3 ? What is your advice?
It really depends on what you want your trainable parameters to be.
For a model such as
a * Input1 * Input 2 + b * Input3 + c
you can use this network
FunctionLayer[
Function[
NetArray[]*#Input1*#Input2 + NetArray[]*#Input3 + NetArray[]]]
If, let’s say, the input are 5-d arrays, the fully specified net will be
d = 5;
net = NetInitialize@FunctionLayer[Function[
NetArray["Dimensions" -> d]*#Input1*#Input2 +
NetArray["Dimensions" -> d]*#Input3 +
NetArray["Dimensions" -> d]],
"Input1" -> d, "Input2" -> d, "Input3" -> d]
THANKS.