# Connecting the Leap Motion controller to Mathematica Using JLink

GROUPS:
 Christopher Wolfram 10 Votes So the Leap Motion controller has just been released and I thought I might as well post some code that Todd Gayley and I wrote to connect the Leap Motion and all of its functionality to Mathematica!Here are the setup instructions:1. (If you haven't already) download and install the leap motionsoftware from leapmotion.com/setup2. Download the leap motion SDK off their developer page.3. Connect the Leap Motion to your computer.4. Find the file "LeapJava.jar" inside of the Leap SDK (It should be under lib.)5. Open Mathematica and paste this in:Needs["JLink"];ReinstallJava[CommandLine -> "java",   JVMArguments ->    "-Djava.library.path=[path to directory CONTAINING LeapJava.jar]"];Now put the path to "LeapJava.jar" in the indicated position, and remember this is the path to the directory containing LeapJava.jar, not the path to LeapJava.jar itself and run the code.6. Now paste this in:AddToClassPath["[path to LeapJava.jar]"];Replace the indicated area with the path to LeapJava.jar itself and run it.7. To setup the controller, run this:controller = JavaNew["com.leapmotion.leap.Controller"]You should now be fully connected and ready to go!  To see if it is working you can run:Methods[controller]and it should return a list of all the methods under the controller.The main method you want to look at is frame[] which contains all of the information about what is going on.To see the methods under frame[] run this:Methods[controller@frame[]]From here you should be able to figure things out like this:controller@frame[]@fingers@count[]which should count the visible fingers in the scene.Also in a final note I want mention that I was having problems accessing the fingers with this:controller@frame[]@finger[finger number]and so you may want to try:controller@frame[]@fingers[]@get[finger number]instead.  The same goes for palms.Well, if you write anything cool using this please post it back up here along with any questions you have.(I am going to try to post some of the code I have written with this soon )
3 years ago
8 Replies
 Trying this out on OS X, Mm 9.0.1.0, I get the following error when setting up the controller:Java::excptn: A Java exception occurred: java.lang.ExceptionInInitializerError       at com.leapmotion.leap.Controller.(Controller.java:39)       at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)       at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)       at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)       at java.lang.reflect.Constructor.newInstance(Constructor.java:513)Caused by: java.lang.SecurityException:           J/Link does not allow code called from Mathematica to call System.exit().       at java.lang.Runtime.exit(Runtime.java:88)       at java.lang.System.exit(System.java:921)       at com.leapmotion.leap.LeapJNI.(LeapJNI.java:18)       ... 11 more.JavaNew::fail: "Error calling constructor for class \!$$\"com.leapmotion.leap.Controller\"$$."I presume this is a permissions issue, but I haven't been able to figure out what file or files permissions need to be changed and to what. I'm running Mathematica from an account with Admin privleges.Any suggestions?Thanks,Mike
3 years ago
 Are you sure that the file path in either of the earlier lines were right?  And that you didn't move any files around inside the leap SDK including files other than LeapJava.jar?  Also you might want to check that you installed the program from the LeapMotion setup page and (I am not sure this is really necessary) have it running.
3 years ago
 I double checked the file paths - first reference is to directory and second is to the .jar file itself. I'm pretty sure I didn't move any of the files around inside the SDK folder - which is one level deep on my desktop. I'm running the Leap app and it connects automatically with other Leap programs. I can successfully compile and run the Leap java Sample app from the terminal, so all seems well there and working nominally.Thanks for the suggestions. I'll keep poking around and see if I can figure out what my problem is.
3 years ago
 Frank Zhao 1 Vote Edit: Found the solution!On Windows (x64), LeapJava.dll from  LeapDeveloperKit\LeapSDK\lib\x64 needs to be in the same directory as your  LeapJava.jar. This should let you run  JavaNew according to the instructions in the original post.------ Disregard below ----Loading the Java class instead of JavaNew worked for me:controller = LoadJavaClass["com.leapmotion.leap.Controller"];Instead of:controller = JavaNew["com.leapmotion.leap.Controller"]But this only lets you view the methods, and does not create an object of the class.
 Rolf Mertig 1 Vote How could I improve the swipe detection robustness? Should I change the frame rate? Or is there some other parameter I should change?This at least does only work sometimes:  (* this is the directory where LeapSDK is *)     $leapdir = FileNameJoin[{$UserBaseDirectory, "Applications",             "LeapDeveloperKit"}];         If[FileExistsQ[$leapdir], Needs["JLink"]; Block[{nativelibs}, With[{(*the location of the developer kit:*) lmlib = FileNameJoin[{$leapdir, "LeapSDK", "lib"}]},             nativelibs =             FileNameJoin[{lmlib,               Switch[$SystemID, "Windows-x86-64", "x64", "Windows", "x86"]}]; (*now start Java*) JLinkReinstallJava[ JLinkClassPath -> FileNameJoin[{lmlib, "LeapJava.jar"}], JLinkJVMArguments -> "-Djava.library.path=\"" <> nativelibs <> "\""]]]]; controller = JavaNew["com.leapmotion.leap.Controller"]; LoadJavaClass["com.leapmotion.leap.Gesture$Type"];        If[Not[controller@isGestureEnabled[Gesture$TypeTYPEUSWIPE]], controller@enableGesture[Gesture$TypeTYPEUSWIPE]          ];        controller@isGestureEnabled[Gesture\$Type`TYPEUSWIPE];    gesturePrint[controller_] := Module[{gesture},      gesture = controller@frame[]@gestures[]@get[0];      If[(gesture@type[]@name[]) === "TYPE_SWIPE",       swipe = JavaNew["com.leapmotion.leap.SwipeGesture", gesture];       {swipe@state[]@toString[],        (swipe@position[]@#) & /@ {getX[], getY[], getZ[]},        (swipe@direction[]@#) & /@ {getX[], getY[], getZ[]},        swipe@speed[]        }]]    ud = .05;    Dynamic[gesturePrint[controller], UpdateInterval -> 0.1]