Message Boards Message Boards

GROUPS:

Identify emotions from a drawn face using machine learning?

Posted 3 months ago
669 Views
|
5 Replies
|
3 Total Likes
|

Can you advise me what kind of machine learning to achieve? ... I had thought, for example, to make a face and, by varying the expression through the "manipulate" function (I do not know if it is possible to do so), the machine will tell me if it is sad, happy, etc. This is the code that allows me to manipulate the face:

Manipulate[
 eyeMat = {{1/(eyeRadius - pupilRadius/2), 0}, {0, 1/(0.15 + eyes - pupilRadius/2)}};
 If[Norm[eyeMat.(pup - eyeCenter[[left]])] < 1, 
  pupNow = pup - eyeCenter[[left]];];
 If[Norm[eyeMat.(pup - eyeCenter[[right]])] < 1, 
  pupNow = pup - eyeCenter[[right]];];
 Graphics[{face, eye[left, eyes], eye[right, eyes], Blue, 
   pupil[left, pupNow, eyes], pupil[right, pupNow, eyes], Black, 
   browDraw[left, brows, eyes], browDraw[right, brows, eyes], 
   Inset[mouthDraw[mouth], {0, -0.5}]}, 
  ImageSize -> {400, 450}], {{brows, {-Pi/20, 0}}, {-0.6, 0}, {0.6, 
   0.15}, ControlPlacement -> Left}, {{eyes, 0}, -0.07, 0.07, 
  ControlPlacement -> Left, 
  ControlType -> VerticalSlider}, {{mouth, 0.15}, -0.401, 0.4, 0.01, 
  ControlPlacement -> Left, 
  ControlType -> VerticalSlider}, {{pup, {0, 0}}, Locator, 
  Appearance -> None}, 
 Initialization :> (face = Circle[{0, 0}, {1, 1.2}]; pupNow = {0, 0};
   left = 1; right = 2;
   eyeRadius = 0.18; eyeCenter = {{-0.4, 0.15}, {0.4, 0.15}};
   pupilRadius = 0.09;
   browUp = 0.25; browW = 0.2; browAng = Pi/20;
   eye[side_, eccen_] := {Black, 
     Circle[eyeCenter[[side]], {eyeRadius + 0.05, eyeRadius + eccen}]};
   pupil[side_, pup_, 
     eccen_] := {Disk[eyeCenter[[side]] + pup + {0, pup[[2]] eccen}, 
      pupilRadius + Max[0, eccen/3]], Black, 
     Disk[eyeCenter[[side]] + pup + {0, pup[[2]] eccen}, 0.03]};
   browDraw[side_, {browAng_, browLift_}, eccen_] := 
    Rotate[{Thickness[0.01], 
      Line[{{eyeCenter[[side]] + {-browW, 
           browLift + browUp + 0.5 eccen}, 
         eyeCenter[[side]] + {browW, 
           browLift + browUp + 0.5 eccen}}}]}, 2 (side - 1.5) browAng];
   mouthDraw[s_] := 
    ParametricPlot[{Cos[u], -s Sin[u]}, {u, Pi/6, Pi - Pi/6}, 
     Axes -> False, PlotStyle -> {Red, Thickness[0.02]}, 
     PlotRange -> All];)]

Now, How can I teach the machine to say what kind of expression the face has (sad, happy, etc.)? Help please.

5 Replies
Posted 3 months ago

The in-build facial expression classifier was trained on human faces so it does not do a good job on simple drawings. I changed the displayed part of the Manipulate to

Column[{g = 
   Graphics[{face, eye[left, eyes], eye[right, eyes], Blue, 
     pupil[left, pupNow, eyes], pupil[right, pupNow, eyes], Black, 
     browDraw[left, brows, eyes], browDraw[right, brows, eyes], 
     Inset[mouthDraw[mouth], {0, -0.5}]}, ImageSize -> {400, 450}], 
  Classify["FacialExpression", g]}]

and Increased the thickness of the lips and eyebrows. Here some examples of the results.

To get better results you would have to generate a dataset of classified drawings and train a neural network on them.

enter image description here

enter image description here

enter image description here

Posted 3 months ago

Thanks for the help, could you tell me how I could make a DataSet?

Posted 3 months ago

One option is to add a dropdown selection control to the manipulate that is populated with the various emotions you want to classify, e.g. Happy, Sad, Afraid, Angry, Neutral..., and a Save button that exports the Graphic to a file in a folder named after the selected emotion. Then run the Manipulate, adjust the facial feature controls, select the appropriate emotion from the dropdown and click the save button. Repeat until you have several examples for each emotion. Then use maybe 80% of the classified data for training a neural network and the remaining 20% for testing. Consult the documentation to learn how to build and train a network.

Posted 2 months ago

Could you please kindly illustrate me something? .... I do not know where to start.

Making the interface using the Chernoff faces package.

Import["https://raw.githubusercontent.com/antononcube/\
MathematicaForPrediction/master/ChernoffFaces.m"]

Manipulate[
 Column[{g = 
    ChernoffFace[<|"ForeheadShape" -> face[[1]], 
      "EyesVerticalPosition" -> face[[2]], "EyeSize" -> eyes[[1]], 
      "EyeSlant" -> eyes[[2]], "LeftEyebrowSlant" -> brows[[2]], 
      "LeftEyebrowTrim" -> brows[[1]], "MouthSmile" -> mouth[[2]], 
      "MouthWidth" -> mouth[[1]]|>, ImageSize -> {450, 450}], 
   Classify["FacialExpression", g]}],
 {{face, {0.5, 0.5}}, {0, 0}, {1, 1}, ControlPlacement -> Left},
 {{eyes, {0.5, 0.5}}, {0, 0}, {1, 1}, ControlPlacement -> Left},
 {{brows, {0.5, 0.5}}, {0, 0}, {1, 1}, ControlPlacement -> Left},
 {{mouth, {0.5, 0.5}}, {0, 0}, {1, 1}, ControlPlacement -> Left}
 ]

enter image description here

Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract