Message Boards Message Boards

[WSS17] You talkin' to me?

Robert De Niro famously asked "You talkin' to me?" in the classic 1976 film "Taxi Driver": enter image description here

Perhaps this was meant rhetorically, but little did he suspect that he would only have to wait some 40 years before neural networks answered the question. Let's break it down with examples:

No, Ironman is not talkin' to you, he ain't even lookin' at you!

enter image description here

Yeah, now he may be talkin' to you, so what?

enter image description here

Now it looks like he's lost interest, get over yourself:

enter image description here

Ok, so where do the neural networks come in? And why is Ironman pixelated and lit up with weird colored lights? We generated lots of images of Ironman lookin' in lots of different directions, and attached to each image specifications for where he is lookin', specifically the cosine and sine of the angle of his gaze. When he's lookin' right at you, it is denoted by {1.00000,0.00000}. When he's lookin' straight to the right, it's denoted by {0.00000,1.00000}, and so on.

enter image description here

Then we used these images to train a neural network to recognize the direction of Ironman's gaze. Well, these images along with about 150,000 similar ones. The weird lights coming from different directions help ensure that the neural network is looking at the direction of Ironman's head, not at the direction or color of the ambient lighting. The pixelation (64 by 64 pixels) make the images a more manageable size for computation.
More details may be found in this Mathematica Notebook on GitHub. And if you're interested in talkin' about this, you know where you can find me.

POSTED BY: Jan Segert
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract