Hi,
This is the first time I am using neural networks in Mathematica, and I need some help. My input consists of vectors of 50 elements, with values between [-1, 1] . The output is a value between [0, 1]. I have 1.000.000 vectors for the training set (but I can easily generate more if needed) and 100.000 vectors for the validation set. If I calculate the MeanSquaredLoss for the output of the training set compared to 0.5 (so Total[(trainingout - 0.5)^2]/Length[trainingout]) I get 0.02, so without a neural network. Now, whatever topology I choose, my neural network always quickly converges to the loss of 0.02 that I already begin with. Some of the things I tried are:
NetChain[200, ElementwiseLayer[LogisticSigmoid], 1]
NetChain[200, ElementwiseLayer[Tanh], 100, ElementwiseLayer[LogisticSigmoid], 1]
NetChain[100, ElementwiseLayer[Tanh], 50, ElementwiseLayer[Tanh], 25, ElementwiseLayer[LogisticSigmoid], 1]
The converged neural network (always) seems to return values of around 0.5 for all input vectors, which explains the loss of 0.02. Am I doing something wrong here? What should I change to decrease the loss beyond that I start with?
Thanks, GW