Dear All,
I’m having an issue with Neural Network, something I can’t understand on my own it seems. Working on a situation where 100 elements-vectors are classified in 4 class. I tried to recover from the Classify function the neural network created. I even got pretty good results as seen bellow
Andrea = Classify[TrainingSet, Method -> "NeuralNetwork",
ValidationSet -> TestingSet]
ClassifierMeasurements[Andrea, TestingSet]["Accuracy"]
ClassifierMeasurements[Andrea, TestingSet]["ConfusionMatrixPlot"]
I then naturally tried to recover the network itself, using the following commands :
AndreaNet =
NetReplacePart[
Andrea[[1]]["Model"][
"Network"], {"Output" -> NetDecoder[{"Class", Range[4]}]}]
I can’t quite understand why I can’t, at this point, use the classical ClassifierMeasurements[AndreaNet, TestingSet, "Accuracy"], when I tried it, I got something like :
Max::nord: Invalid comparison with 1.0629 +3.14159 I attempted.
I also tried the ClassifierMeasurements[Classify[AndreaNet], TestingSet, "Accuracy"] without success.
Hence, I programed the evaluation myself (and perhaps in the worst way possible, element after element) as following :
accuracy = 0;
For[i = 1, i <= Length[TestingSet], i++,
print = PrintTemporary[i];
If[AndreaNet[TestingSet[[i, 1]]] == TestingSet[[i, 2]], accuracy++];
NotebookDelete[print]];
accuracy/Length[TestingSet]
% // N
I obtained a value of 5323/8172, which is roughly 0.651
As you can see, I obtain a result way different than what I expected. Can someone help me understand :
- Why can’t I use the function ClassifierMeasurements[AndreaNet, TestingSet, “Accuracy”] here, as I have some example of similar network which works perfectly fine
- Why do the accuracy obtained differs so much from the first evaluation ?
Thank you very much for your attention.
Kind Regards,