Message Boards Message Boards

0
|
2654 Views
|
1 Reply
|
1 Total Likes
View groups...
Share
Share this post:

Question on neural network topology and loss

Posted 2 years ago

Hi,

This is the first time I am using neural networks in Mathematica, and I need some help. My input consists of vectors of 50 elements, with values between [-1, 1] . The output is a value between [0, 1]. I have 1.000.000 vectors for the training set (but I can easily generate more if needed) and 100.000 vectors for the validation set. If I calculate the MeanSquaredLoss for the output of the training set compared to 0.5 (so Total[(trainingout - 0.5)^2]/Length[trainingout]) I get 0.02, so without a neural network. Now, whatever topology I choose, my neural network always quickly converges to the loss of 0.02 that I already begin with. Some of the things I tried are:

NetChain[200, ElementwiseLayer[LogisticSigmoid], 1]
NetChain[200, ElementwiseLayer[Tanh], 100, ElementwiseLayer[LogisticSigmoid], 1]
NetChain[100, ElementwiseLayer[Tanh], 50, ElementwiseLayer[Tanh], 25, ElementwiseLayer[LogisticSigmoid], 1]

The converged neural network (always) seems to return values of around 0.5 for all input vectors, which explains the loss of 0.02. Am I doing something wrong here? What should I change to decrease the loss beyond that I start with?

Thanks, GW

Hi,

I now see that I have not updated this question with the answer. It turned out that my input vectors were 'too random'. The use case was to train a neural network on a number of international draughts positions. If you choose a collection of randomly generated legal positions (a question also is if these positions could be reached at all from the starting position BTW) as the training set the neural network cannot 'find' a relation between the positions and the result (won/draw/lost), so the neural network just predicts the average score (draw). However, if you generate a collection of 'reasonably random' positions by playing a number of games selecting 'reasonable random' moves (so moves that do not immediately loose a man for example) the network starts to 'see' a relation between the positions and the result, and the network predicts about 80-90% of the results correctly.

Regards, GW

Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract