# Identify emotions from a drawn face using machine learning?

Posted 1 year ago
1987 Views
|
5 Replies
|
3 Total Likes
|
 Can you advise me what kind of machine learning to achieve? ... I had thought, for example, to make a face and, by varying the expression through the "manipulate" function (I do not know if it is possible to do so), the machine will tell me if it is sad, happy, etc. This is the code that allows me to manipulate the face: Manipulate[ eyeMat = {{1/(eyeRadius - pupilRadius/2), 0}, {0, 1/(0.15 + eyes - pupilRadius/2)}}; If[Norm[eyeMat.(pup - eyeCenter[[left]])] < 1, pupNow = pup - eyeCenter[[left]];]; If[Norm[eyeMat.(pup - eyeCenter[[right]])] < 1, pupNow = pup - eyeCenter[[right]];]; Graphics[{face, eye[left, eyes], eye[right, eyes], Blue, pupil[left, pupNow, eyes], pupil[right, pupNow, eyes], Black, browDraw[left, brows, eyes], browDraw[right, brows, eyes], Inset[mouthDraw[mouth], {0, -0.5}]}, ImageSize -> {400, 450}], {{brows, {-Pi/20, 0}}, {-0.6, 0}, {0.6, 0.15}, ControlPlacement -> Left}, {{eyes, 0}, -0.07, 0.07, ControlPlacement -> Left, ControlType -> VerticalSlider}, {{mouth, 0.15}, -0.401, 0.4, 0.01, ControlPlacement -> Left, ControlType -> VerticalSlider}, {{pup, {0, 0}}, Locator, Appearance -> None}, Initialization :> (face = Circle[{0, 0}, {1, 1.2}]; pupNow = {0, 0}; left = 1; right = 2; eyeRadius = 0.18; eyeCenter = {{-0.4, 0.15}, {0.4, 0.15}}; pupilRadius = 0.09; browUp = 0.25; browW = 0.2; browAng = Pi/20; eye[side_, eccen_] := {Black, Circle[eyeCenter[[side]], {eyeRadius + 0.05, eyeRadius + eccen}]}; pupil[side_, pup_, eccen_] := {Disk[eyeCenter[[side]] + pup + {0, pup[[2]] eccen}, pupilRadius + Max[0, eccen/3]], Black, Disk[eyeCenter[[side]] + pup + {0, pup[[2]] eccen}, 0.03]}; browDraw[side_, {browAng_, browLift_}, eccen_] := Rotate[{Thickness[0.01], Line[{{eyeCenter[[side]] + {-browW, browLift + browUp + 0.5 eccen}, eyeCenter[[side]] + {browW, browLift + browUp + 0.5 eccen}}}]}, 2 (side - 1.5) browAng]; mouthDraw[s_] := ParametricPlot[{Cos[u], -s Sin[u]}, {u, Pi/6, Pi - Pi/6}, Axes -> False, PlotStyle -> {Red, Thickness[0.02]}, PlotRange -> All];)] Now, How can I teach the machine to say what kind of expression the face has (sad, happy, etc.)? Help please.
5 Replies
Sort By:
Posted 1 year ago
 The in-build facial expression classifier was trained on human faces so it does not do a good job on simple drawings. I changed the displayed part of the Manipulate to Column[{g = Graphics[{face, eye[left, eyes], eye[right, eyes], Blue, pupil[left, pupNow, eyes], pupil[right, pupNow, eyes], Black, browDraw[left, brows, eyes], browDraw[right, brows, eyes], Inset[mouthDraw[mouth], {0, -0.5}]}, ImageSize -> {400, 450}], Classify["FacialExpression", g]}] and Increased the thickness of the lips and eyebrows. Here some examples of the results. To get better results you would have to generate a dataset of classified drawings and train a neural network on them.
Posted 1 year ago
 Thanks for the help, could you tell me how I could make a DataSet?
Posted 1 year ago
 One option is to add a dropdown selection control to the manipulate that is populated with the various emotions you want to classify, e.g. Happy, Sad, Afraid, Angry, Neutral..., and a Save button that exports the Graphic to a file in a folder named after the selected emotion. Then run the Manipulate, adjust the facial feature controls, select the appropriate emotion from the dropdown and click the save button. Repeat until you have several examples for each emotion. Then use maybe 80% of the classified data for training a neural network and the remaining 20% for testing. Consult the documentation to learn how to build and train a network.
 Making the interface using the Chernoff faces package. Import["https://raw.githubusercontent.com/antononcube/\ MathematicaForPrediction/master/ChernoffFaces.m"] Manipulate[ Column[{g = ChernoffFace[<|"ForeheadShape" -> face[[1]], "EyesVerticalPosition" -> face[[2]], "EyeSize" -> eyes[[1]], "EyeSlant" -> eyes[[2]], "LeftEyebrowSlant" -> brows[[2]], "LeftEyebrowTrim" -> brows[[1]], "MouthSmile" -> mouth[[2]], "MouthWidth" -> mouth[[1]]|>, ImageSize -> {450, 450}], Classify["FacialExpression", g]}], {{face, {0.5, 0.5}}, {0, 0}, {1, 1}, ControlPlacement -> Left}, {{eyes, {0.5, 0.5}}, {0, 0}, {1, 1}, ControlPlacement -> Left}, {{brows, {0.5, 0.5}}, {0, 0}, {1, 1}, ControlPlacement -> Left}, {{mouth, {0.5, 0.5}}, {0, 0}, {1, 1}, ControlPlacement -> Left} ]