Message Boards Message Boards

[WSS18] Parental kinship recognition


Parental kinship recognition


Facial images convey many important human characteristics, such as identity, gender, expression, age, and ethnicity. Compared with these face analysis tasks, facial kinship verification is a relatively new research topic in face analysis and only some attempts have been made over the past few years. However, this new research topic has several potential applications such as family album organization, image annotation, social media analysis, and missing children/parents search.


The objective of the project I realized in the Wolfram Summer School 2018 was to identify parental relationships using faces photographs.


The database used is the KinFaceW includes around 50 % Caucasians, 40 % Asians, 7 % African Americans, and 3 % others. KinFaceW-II contains 250 pairs for each category ("Father-Son","Father-Daughter","Mother-Son","Mother-Daughter").

fatherSon = Import["Assets/KinFaceW-II/images/father-son/fs*"];
fatherDaughter = Import["Assets/KinFaceW-II/images/father-dau/fd*"];
motherSon = Import["Assets/KinFaceW-II/images/mother-son/ms*"];
motherDaughter = Import["Assets/KinFaceW-II/images/mother-dau/md*"];

KinFaceW-II images

Augmented data

The following function was applied to crop the faces and augment the dataset.



Feature extraction

I use ResNet 101 feature extractor to build a multilayer transfer learning neural network.

convP = NetModel["ResNet-101 Trained on Augmented CASIA-WebFace Data"];

Features were extracted before the training in order to reduce the computational usage.

fmodel = NetGraph[{convP, convP, 
                                {NetPort["Parent"] -> 1 -> 3, NetPort["Child"] -> 2 -> 3}];

lnet = NetInitialize[fmodel] 

vectorsAugmentedFatherSon = lnet[dataAugmentedFatherSon];
vectorsAugmentedFatherDau = lnet[dataAugmentedFatherDau];
vectorsAugmentedMotherSon = lnet[dataAugmentedMotherSon];
vectorsAugmentedMotherDau = lnet[dataAugmentedMotherDau];

3D Plot of ResNet 101 features for KinFaceW-II

Neural Network

vectorsNet = NetGraph[{LinearLayer[256], LogisticSigmoid , LinearLayer[], 
                                    SoftmaxLayer[]}, {NetPort["Input"] -> 1, 1 -> 2 -> 3 -> 4}, 
                                    "Input" -> {4096}, 
                                   "Output" -> NetDecoder[{"Class", 
                                                            {"FatherSon", "FatherDau", "MotherDau", "MotherSon"}}]];
vectorsNet = NetInitialize[vectorsNet]

fmodelValidationAugmented = NetGraph[{convP, convP, CatenateLayer[], 
                                                                vectorModelAugmented}, {NetPort["Parent"] -> 1 -> 3, NetPort["Child"] -> 2 -> 3, 3 -> 4}, 
                                                               "Output" ->NetDecoder[{"Class",
                                                                                     {"FatherSon", "FatherDau", "MotherDau", "MotherSon"}}]];

netValidationAugmented = NetInitialize[fmodelValidationAugmented]

After 100 rounds of trainings the results are:

cm = ClassifierMeasurements[netValidationAugmented, test]

Confusion Matrix

An accuracy of 87% was achieved, that is not bad for state of the art kinship recognition algorithms.

Finally, I did an extra validation just to check that the trained net is working.

Extra validation

Future work

  • Increase the dataset in order to obtain better results.
  • Train the model for siblings and grandparents
  • Determine the best traits for classification.
POSTED BY: Said Munoz Montero
5 days ago

Group Abstract Group Abstract