A tiny startup outside Chicago has created external hardware for a smartphone that enables a user to feel as well as see an image on flat glass. The user has the sensation of touching keys on a keyboard, for example.
The company, called Tanvas, is selling its $2,000 TPad phone, a standard Motorola Moto G coupled with haptic technology, which creates the illusion of touching physical textures and shapes displayed on its screen. "This is a new medium; a new way to display information," says Joe Mullenbach, a Northwestern University Ph.D. candidate and one of two developers of the TPad, or tactile pattern display, phone.
A crude version of a haptic interface already comes with smartphones in the form of vibrations that alert you to an incoming call or text message in place of a ring tone. "Everything today is audio, visual or both, but people actually get a lot of information from tactile experiences," Mullenbach says.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
The TPad looks almost exactly like a Moto G snuggled in a protective case. Tanvas's black hardware weighs 50 grams, most of which is the plastic-and-glass housing. A plate of glass covers the phone's screen, and the plastic housing surrounds the rest. The haptics hardware requires relatively little power—about the equivalent of playing the Moto G's speakers at full volume.
The technology works by tricking the brain into feeling something that doesn't exist, not unlike the way a television fools the eye into perceiving motion. A tiny actuator in the housing makes the glass resonate at 38.7 kilohertz, an ultrasonic frequency that cannot be heard or felt. Tanvas’s algorithms analyze images on the Moto G's screen, and the phone calculates where over the screen the user's finger is positioned. These data are translated into ultrasonic oscillations that create the illusion of something physical.
The ultrasonic waves repel a fingertip from the glass. Larger-amplitude waves impede contact, making the screen feel slippery. Smaller amplitude oscillations allow more contact and therefore produce more friction.
There might, for instance, be an image on the screen of a roadway rumble strip that conveys a bumpy sensation to warn a driver not to drive out of a lane. A finger running over a valley in the strip would experience little friction and easily slide to an area that generates more friction, which would be sensed as one of the ridges in the rumble strip.
The TPad is capable of simulating smooth and raised shapes, abrupt edges and curved lines. Bas-relief images on, say, a coin can be faithfully represented. A scanned page's words and images might be made to feel as if they were on a raised surface. (See video for a more complete description of how the TPad phone works.)
"I have tried it. It's a fairly good idea,” says Vincent Hayward, a professor with the Institute for Intelligent Systems and Robotics in Paris. Hayward is not convinced, though, that the sensations provided by the TPad’s friction-modulation technique will be useful for people with visual impairments.
"There's some debate about exactly what is happening," Hayward says. He said the waves might be causing the air immediately above the screen to act like a cushion, "buffeting your finger" and preventing it from touching the surface.
Research into haptics goes back to the 1860s. Work on glass screens, in particular, began to grow in the last decade. Four Northwestern researchers described a tactile pattern display in a 2007 paper (pdf). Two of the researchers, mechanical engineering professors Ed Colgate and Michael Peshkin, founded Tanvas in 2010.
A separate effort by a French research team at the University of Lille has come up with a similar technology. And Tanvas—which in June received $5 million in funding from investment firms R7 Partners and Peak6, and from Northwestern itself—is exploring a new way of providing touch sensations on the screen, a technique that attracts rather than repels fingertips to simulate shapes.
Work on the TPad technology began about three years ago, resulting first in a tablet version and then a phone. Mullenbach’s team focused on a commercial haptic phone about six months ago. They sent working versions to 10 research teams around the world, and these teams developed and displayed new apps for the phone at the June World Haptics Conference at Northwestern.
One version, created by researchers from Scuola Superiore Sant’Anna in Italy, enables a person who is blind or visually impaired to compose a photo by feeling simple images of people on the screen. A Brazilian team at Universidade Federal do Rio Grande do Sul used the TPad to create a new level of password security by enabling a person to feel for the correct combination.
Tanvas has assembled about 30 TPad phones, all by hand. They’ve sold 20 of them, and most of the rest have been given to other researchers.