I tried the fix that Jamie suggested and it does eliminate the error. However I don't think it explains why training on a subset of the training data also works even though the subset contains even fewer unique values.
Convert to numeric
classifier = Classify[N[training]]
classifier[testing]
All testing samples are classified as P, is that expected?
"P", "P", "P", "P", "P", "P", "P"}
The probabilities for P and NP are extremely close for most of the samples.
classifier[testing, "Probabilities"] // Column
{
{<|"NP" -> 0.496296, "P" -> 0.503704|>},
{<|"NP" -> 0.486345, "P" -> 0.513655|>},
{<|"NP" -> 0.499325, "P" -> 0.500675|>},
{<|"NP" -> 0.494909, "P" -> 0.505091|>},
{<|"NP" -> 0.492273, "P" -> 0.507727|>},
{<|"NP" -> 0.480284, "P" -> 0.519716|>},
{<|"NP" -> 0.458435, "P" -> 0.541565|>}
}
I tried several of the available classification methods to see how much the results varied
methods = {"DecisionTree", "GradientBoostedTrees",
"LogisticRegression", "NaiveBayes", "NearestNeighbors",
"NeuralNetwork", "RandomForest", "SupportVectorMachine"};
classifiers = Map[{#, Classify[N[training], Method -> #]} &, methods];
Map[First[#] -> Last[#][testing] &, classifiers] // Column
Gives
{
{"DecisionTree" -> {"NP", "NP", "NP", "P", "NP", "NP", "P"}},
{"GradientBoostedTrees" -> {"P", "P", "NP", "NP", "NP", "P", "NP"}},
{"LogisticRegression" -> {"P", "P", "P", "P", "P", "P", "P"}},
{"NaiveBayes" -> {"NP", "P", "NP", "NP", "NP", "P", "P"}},
{"NearestNeighbors" -> {"NP", "NP", "NP", "NP", "NP", "NP", "NP"}},
{"NeuralNetwork" -> {"P", "P", "P", "NP", "P", "P", "P"}},
{"RandomForest" -> {"NP", "NP", "NP", "NP", "NP", "NP", "NP"}},
{"SupportVectorMachine" -> {"NP", "NP", "NP", "NP", "NP", "NP", "NP"}}
}
Without knowing the details of what this data represents it is hard to know which method is most appropriate.