When Tom Cruise’s character in the movie “Minority Report” controls a computer only by waving his hands, it looks really cool.
But in the real world, using touchless gestures to interact with a PC can be a study in frustration. That was my take after testing the Leap Motion Controller, a new device from a San Francisco startup that brings gesture controls to Macs and Windows-based PCs.
Launched last summer, the Controller is an $80 device about the size and shape of USB flash drive. It has a pair of cameras that point upward and detect hand motions made in a cone of space over the device. Employing particular gestures, users can open applications, scroll through documents, rotate virtual 3-D objects and zoom in on pictures.
In some ways, it’s like a miniaturized and less costly version of Microsoft’s Kinect, but more limited in range and capabilities. Where Kinect can track an entire body, the Controller detects only hand and finger motions. And where Kinect can detect motion from about a 10-foot distance, the Controller’s range is about 2 feet.
The Controller is easy to set up. It connects to a computer via a USB cable, which also provides its power. To get it to work, you download and install some software from Leap Motion’s website.
Leap’s software provides a link to the company’s app store, which offers about 100 gesture-controllable programs.
Among them, you’ll find games, including a version of the popular smartphone title “Cut the Rope”; programs that let you play virtual musical instruments; and educational applications that let you do things like dissect a virtual frog. You’ll also find programs that work at a more basic level, allowing you to launch traditional programs or scroll through Web pages using gestures.
Some of these apps are clever and compelling. Cyber Science’s Motion app allows you to rotate a virtual human skull, take it apart and examine it piece by piece. With Vito Technology’s Solar Walk, you can take a virtual tour of the solar system, zooming in on particular planets and moons or panning way out to see the location of our local planetary group within the Milky Way galaxy.
But I found that these apps were often just as compelling and usually much easier to control when using a mouse or trackpad than with gestures.
The Controller can be both hypersensitive and wildly inaccurate when tracking users’ movements. I frequently found myself inadvertently launching programs or selecting menu items when trying to use gestures to do other things. At other times, I found myself repeating gestures over and over, hoping the system would finally respond.
Unlike a touch-sensitive screen or a mouse, the Controller has no definitive point of interaction. Instead, you make your gestures in the free space above the device, hoping they are within the area it can recognize. But because that recognition area is invisible, it’s often difficult to know if you are gesturing within it.
Part of the problem with trying to figure out where to make the gestures is that there’s no designated place to locate the device. Leap’s diagrams suggest that it can be located in front of, behind or to the side of a keyboard. I tried it in all three places and never found one that noticeably improved my accuracy.
I did find that my ability to interact using gestures improved the more I used the Controller. But it still was far less reliable than a touch screen or a mouse.
Some of the apps help compensate for the imprecision of the gesture controls. With Emantras’ Frog Dissection app, for example, I was able to stick pins in the appropriate places and cut along the prescribed lines even though I couldn’t move my finger to the exact locations or pantomime anything close to a straight line.
But that app is more of an exception than a rule. Most of the apps I tested were far less forgiving. To work properly, they required the precision of a fingertip or even a mouse pointer, rather than the vagary of a free-floating gesture.
Another big problem with the Controller is that Leap hasn’t set any strict guidelines or standards for the gestures you use to control apps. So you end up having to learn a different set for each app you launch.
With some, you might select a menu item by simply pointing at it for a second or two. With others, you might do so by curling and then extending your index finger. In still others, you might need to point at the screen with your thumb extended, then touch your thumb to your index finger.
So it can be confusing and frustrating to have to remember which is the right gesture to use.
Somehow I don’t remember Cruise’s character experiencing similar struggles. We may someday end up with a touchless computer like that in the “Minority Report,” but the current iteration of Leap Motion’s Controller doesn’t get us there.