(Note – This article was first published by h+ Magazine, on May 12, 2010. It’s republished here for my own archives.)
Chances are, you’re not using the same computer you were twenty years ago. But chances are, you’re still using the same basic user interface — a mouse for pointing, a keyboard for typing. While generations of hardware and software have come and gone, the paradigm for interacting with our machines has remained pretty much the same. But that’s slowly changing: Nintendo’s Wii system has gotten video gamers off the couch, Apple’s iPhone has acclimated us to using touch to control our devices, and a new generation of user-interface systems are beginning to come to the consumer market which promise more natural, intuitive, and engaging experiences.
Vincent John Vincent, president and co-founder of GestureTek, thinks next-generation user interfaces will be driven by movement. His company makes systems that use cameras and computer vision to watch a user’s movements, and then translate those movements into controls. “As the interfaces that we see on the screen become more dynamic and deep, with 3-D jumping off the screen or deeper into the screen, then the ability to reach out with your hand and manipulate them in that 3-D space is a much more natural way to go than just to have one point of control with the mouse,” Vincent said.
GestureTek has been building equipment and displays that respond to movement since the 1990s, according to Vincent. “We are the inventors and pioneers in this space, and luckily, early enough in it as well that we’ve been able to get a lot of patents on what we’ve done,” he said. “We’re very lucky that we were way ahead of our time for a long time, we sold thousands of installations of that technology, into museums and science centers and retail in various locations.”
“In the early 2000s, we started expanding and we created a number of different technologies that were interactive surfaces like floors and walls and windows and whatnot that would just be reacting to your motion and movement and being able to let people walk over those in front of them and just pick up your general gestures,” Vincent said. “We just found it to be very, not just natural, but engaging. It captures people’s attention, it makes the experience more entertaining and dynamic.”
And gesture controls are coming to a screen near you sooner than you might think — Microsoft’s Project Natal, which was announced at last year’s Electronic Entertainment Expo, is due to be released by Christmas of this year. Project Natal promises a 3-D, depth-sensing camera peripheral for the Xbox 360, which will use software licensed from GestureTek to enable gamers to play without any controller at all, simply by using the movements of their body. Hitachi has demonstrated a prototype HDTV that uses an embedded 3-D camera to replace the traditional remote control. Users simply wave their hand to change the channel or turn down the volume. Mobile phone manufacturers such as DoCoMo have licensed software from GestureTek as well. “We looked at the mobile phone market and said well, there is a processor, a display and a camera, what a perfect product,” said Vincent. “So we took what we had already created and evolved it even more so that we could use a phone in the hand to act like a joystick, and the camera would watch how the phone was moving in relationship to the world, or you could gesture at the phone, etc.”
The availability of depth-sensing cameras is driving the speed with which these innovations are getting into the hands of consumers. “The depth cameras that are coming to the market use infrared light and three depth sensors, so that when an array of light is pulsed out into the environment, much like an ultrasound, it bounces back,” Vincent said. “The sensors can tell how quickly it’s come back and therefore build up a depth perception of the world in front of it. It’s very similar to ultrasound but done with light. Obviously, the lenses that are capturing that information are very sophisticated and that is what has kept them very expensive up till this point in time.”
Of course, those hardware costs will drop over time. And that means one day soon, you might be able to ditch that mouse once and for all.