New Tablet Case Recognizes Sign Language and Translates It Into Text

MotionSavvy

When you’re deaf, finding a job isn’t easy.

The trickiest part, explains Ryan Hait Campbell, is the interview. “You’re not required to tell an employer you’re deaf until the interview, but sometimes, they’re a little shocked,” says Campbell, who has been deaf since birth. “They don’t know how to handle it.”

Because of things like this, he says, unemployment rates are staggeringly high among the deaf. Hard numbers are tough to come by, but some figures estimate that around half of people with hearing disabilities are unemployed.

But Campbell wants to change this. He’s the co-founder and CEO of MotionSavvy, an Alameda, California-based startup that’s developing a case for tablet computers that can serve as a virtual interpreter for the deaf. Known as UNI, the case uses gesture recognition technology developed by Leap Motion to translate sign language into audible speech. It then merges this with voice recognition technology to convert spoken word to text. Because there are a variety of signs for any given word, users can upload new signs using a feature called Sign Builder. The system learns how individual users sign, while also distributing each new sign to every UNI device.

‘This could really give deaf people the power to live the lifestyle they want to live. We think that is very powerful.’

On Tuesday, MotionSavvy launched an IndieGoGo campaign for UNI to raise money and recruit beta testers to help build its dictionary of signs. For $499, a discounted rate, 200 selected backers will get a tablet and UNI case to try at home. “This could really give deaf people the power to live the lifestyle they want to live,” Campbell says, “and we think that is very powerful.”

Such technology would have seemed a distant dream not long ago. But the past decade has brought a wave of investment and interest in both gesture recognition technology and voice recognition technology, driven by companies like Apple and Microsoft, as well as smaller players like Nuance and Leap Motion. That hasn’t gone unnoticed by those who want to improve the lives of the deaf community. MotionSavvy is one of several players trying to capitalize on the convergence of these trends.

Just last week, Transcense, launched an IndieGoGo campaign for an app that provides real-time voice recognition so deaf people can follow a conversation. But unlike UNI, it doesn’t give deaf people who haven’t mastered speech a clear way to talk back. For MotionSavvy, that is the final—and most important—puzzle piece.

“It’s kind of like solving a quadratic equation at this point. It’s figuring out the right variables and stacking things together in such a way that they’ll all perform efficiently,” says Stephen Jacobs, associate director of Rochester Institute of Technology’s Center for Media Arts, Games Interaction, and Creativity.

Jacobs introduced Campbell to MotionSavvy CTO Alexandr Opalka when both were studying at RIT. Opalka, who also is deaf, had been working on similar technology as a student in RIT’s National Technical Institute for the Deaf. They teamed up with four other deaf students, and in 2012, launched MotionSavvy.

The technology is in its earliest stages. UNI recognizes only 300 signs, and its voice recognition component remains unreliable, though Opalka says UNI will come equipped with new and improved voice recognition for beta testers. And yet, during a demo of UNI at WIRED’s New York City office, it wasn’t hard to see just how transformative a technology like this could be. Campbell used it to sign a few common phrases to Opalka, such as “What’s your name?” and “Where are you from?” Yes, it was wonky, but still it struck me as sort of magical.

‘I brought this to a table of older deaf people, and they all freaked out.’

Campbell says that reaction’s not entirely unique. “I brought this to a table of older deaf people, and they all freaked out,” he says.

But it’s not just the deaf and hard of hearing who are excited about UNI. Campbell says the FCC has gotten in touch. For many low-income deaf people, translators, video relay services, and other communication tools are prohibitively expensive. So the National Deaf-Blind Equipment Distribution Program picks up the tab. When the commercial version of UNI launches in 2015, it’ll cost $799, plus a $20 monthly subscription for Sign Builder. It’s not cheap, but it’s better than the alternative.

Campbell acknowledges the product is a “moonshot,” and admits it may never replace human interpreters. In fact, he and Olapka hope that it does the opposite. If UNI can achieve its intended purpose—facilitating one-on-one communication— then it could become easier for deaf people to get decent jobs. And who typically pays for interpreters? Employers. “If you can’t communicate during an interview, you’re not getting the job,” Olapka says. “With UNI, we predict more people who are deaf will be able to get jobs and stay working, and that’s how we’ll get more people to hire interpreters. There will be more people in the workforce.”