Hi Francisco,

Can you explain what you mean by

but this was obviously too heavy a task for it.

• What is the subject in the images?
• Are the images very similar even though they are labeled differently?
• How large is the training set?
• Is there a large class imbalance in the training set?

Have you looked at the image classification examples in the Applications section of the documentation? It performs well on those images.

Could you post a few example images from each class?

Many thanks for your questions. The answers are the following:

a. What is the subject in the images? These are photos of people who belong to different non-state armed groups (guerrillas or militias)

b. Are the images very similar even though they are labeled differently? I frankly do not know. This is the very sense of trying to find a classifier that is able to tell them apart. My hypothesis is that they are different enough to be separated by a good classifier.

c. How large is the training set? I used 400 pics for each group --that means 800 in total. Maybe this is too small?

d. Is there a large class imbalance in the training set? No imbalance

I am attaching two examples (one for each group) of the type of photos I am dealing with (by the way, retrieved with WebImageSearch).

I will give a hard look at the Classification function documentation, though I have already know it (I think) quite well --thanks again

I am confused by the response to question b.

I frankly do not know. This is the very sense of trying to find a classifier that is able to tell them apart. My hypothesis is that they are different enough to be separated by a good classifier.

In supervised learning the training data has to be accurately labeled (ground truth) as in the examples in the documentation. How were the images labeled?

Something to try.

To get a random 80%, 10%, 10% split for training, validation and testing. images is the list of image -> class

{training, testing} = ResourceFunction["TrainTestSplit"][images];
{testing, validation} = ResourceFunction["TrainTestSplit"][testing, "TestSetSize" -> Scaled[0.5]]

classifier = Classify[training, ValidationSet -> validation, PerformanceGoal -> "Quality"]
cm = ClassifierMeasurements[classifier, testing]

cm["ConfusionMatrixPlot"]
cm /@ {"Accuracy", "Precision", "Recall", "F1Score"}

If the results are not satisfactory there are a lot of things to try to improve it. I would do that rather than building a DNN from scratch or even attempting transfer learning.

There is no need to write a NN from scratch in WL. It has extensive built-in support for NN.

Writing a good image classifying NN from scratch is not an easy task.