On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
Our homes will get a lot smarter in the coming years, allowing us to use a smartphone to manage an integrated system of appliances and other electronics from any room—a kind of universal remote control for the “Internet of things.” An even easier, more intuitive approach, however, would allow us to flip through TV channels or turn on the coffeemaker with a simple hand wave.
A team of University of Washington (U.W.) computer scientists is developing just such a system (pdf). Dubbed “WiSee,” the system uses a modified wireless router to receive and identify slight disturbances in wi-fi signals that permeate living spaces. Software algorithms interpret the hand or body gestures that create those disturbances and translate the movements into commands for controlling specific wireless devices.
In-air gesture recognition has already been popularized by Microsoft’s Xbox Kinect game console. But Kinect is a vision-based system that relies on a camera to pick up the gestures players make; someone out of the camera’s line of sight is unable to interact. Because wi-fi signals can travel through walls, WiSee would let users control devices in different rooms from anywhere inside the household. In a sense, it repurposes wi-fi signals that already exist to perform gesture recognition—no need to buy motion-sensing cameras.
When a person moves in an environment saturated with wi-fi signals from routers, laptops and mobile phones, that movement creates a slight change in the frequency of those signals, says lead researcher Shyam Gollakota, a U.W. assistant professor of computer science and engineering. Leveraging this Doppler shift—the change in the frequency observed by the transmitter and the receiver as disturbed waves move relative to those locations—the researchers designed the WiSee receiver to understand signal disturbances created by different motions of a person’s arm, leg or entire body. Ideally, a person sitting in his or her living room on a hot summer day could crank up the air-conditioning using, well, a cranking motion in the air. Likewise, a person engaged in a conversation in one room could lower the TV volume coming from another room simply by patting the air in a downward direction.
To test the system in a real-world environment, the researchers tracked five volunteers operating first in an office setting and then in a two-bedroom apartment. The people were instructed to make nine specific gestures with their bodies, including pushing, pulling and kicking motions. Out of the 900 total gestures performed during the test, WiSee responded correctly 94 percent of the time. A four-antenna version of the WiSee receiver tested inside the apartment could identify gestures from a primary user even when other people were nearby and performing random gestures. Accuracy declined, however, as the number of distractions increased.
Each primary user during the test identified himself or herself using a particular “preamble” gesture that WiSee was programmed to identify, Gollakota says. “One can also imagine a system where each user can set [his or her] own preamble to achieve some form of [secure] access control,” he adds. Such identification and control measures would prevent your neighbor from using wi-fi signals to turn down your stereo from his own apartment. They would also prevent you from, say, shutting off the lights every time you attempt to swat a fly.