A Touch-Free Smartphone the Disabled Can Control With Their Heads

CEO Oded Ben Dov with phone user. Sesame Enable

Shortly after appearing on Israeli television with a new computer game you control merely by moving your head, Oded Ben Dov got a phone call.

It came from a complete stranger who just happened to see this TV appearance, and he had a question. “I wasn’t sure if it was a prank call or not, but then he started to say some serious stuff,” Ben Dov remembers. “So I listened.”

Screenshot: Sesame Enable

The man on the other end of the line was Giora Livne, a former Israeli navy commander and electrical power engineer. He’d been quadriplegic for seven years, he explained, which made it impossible to use a smartphone without help. That meant, among other things, seven long years without a private phone call or email or text message—seven years without much privacy at all.

Then Livne asked for help. “Could you make a smartphone that I could use?” he asked.

It’s taken two years, but Ben Dov has done just that. The Sesame Phone is a smartphone designed specifically for the mobility impaired. It uses computer vision technology to allow someone like Livne to access any app simply by moving his head. The phone is from Sesame Enable, a startup Ben Dov and Livne founded to bring the idea to life, and today the company is launching an Indiegogo campaign to raise the $50,000 needed to continue developing the phone.

The Sesame phone uses computer vision technology to allow someone like Livne to access any app just by moving his head.

Sesame Phone is the latest example of how gesture and facial recognition technology are turning things like smartphones and tablets into vital communication devices for people with special needs. Just last month startup MotionSavvy launched an Indiegogo campaign for a device that translates sign language into spoken word.

Many of the companies, like Microsoft and Intel, that have pushed this recognition technology along have done so for the general consumer. They see gesture recognition as a way to make the computing experience more immersive while removing a level of friction between man and machine. Yet all of this work is culminating in entirely new ways for disabled people to interact with gadgets the rest of us take for granted.

According to research from the Christopher Reeve Foundation, about six million people are living with some form of paralysis. Of that six million, a little more than two million report having a lot of difficulty moving, while one million of them say they’re unable to move at all—and that’s just in the US. These are the people Ben Dov wants to help.

Open Sesame

The Sesame Phone is a Google Nexus 5 phone customized to include facial recognition capabilities and a cursor that floats over the home screen, allowing people to click on apps without using their hands. Users open the Sesame Enable app by saying, “Open Sesame,” which opens a window that capture’s key points on the user’s face.

Sesame Enable

When the phone has a good read of a user’s face, a cursor appears. Turn your head slightly to the right and the cursor moves right. Stop, and a navigation icon appears, allowing you to click, drag, swipe, or exit.

During a demo at WIRED’s office in Manhattan, Ben Dov made what seemed like a telepathic call to his wife in Israel. I used it to open Chrome, and Ben Dov says one young tester recently used it to play Angry Birds for the first time. “At one point he went into my account and tried purchasing more birds. I was like: ‘Hey!’,” he says. “It was impossible to take the device away from him.”

A Long Way to Go

That said, Ben Dov admits the Sesame Phone has a long way to go. It crashed repeatedly during the demo, and while Livne can use it, the company still needs to ensure the phone works for a variety of disabilities.

“One big challenge is making sure it will be accessible to as many people as possible,” says Sharon Besser, one of Sesame Enable’s investors. The system will have to recognize, for instance, the broad sudden movements of people with cerebral palsy as effectively as the slight, slow movements of people with severe spinal cord injuries. Down the line, Besser says, it will also be important for Sesame Enable to accommodate people with severe ALS, who can’t move their heads at all, using vision-tracking technology.

Meanwhile, Ben Dov also is working with smartphone manufacturers to integrate Sesame Enable’s technology into existing phones. That not only will expand the options that disabled customers have, but it could also bring down the price of the phone, which now costs a hefty $900. In the meantime, he’s developed an SDK that other app developers can integrate into their own products. Rovio, for instance, could build face recognition capability right into Angry Birds, so that users could use any phone to control the game with their heads.

Ben Dov knows there could be plenty of other applications for technology like this in industries like gaming. And yet, he wants to perfect it with the special needs market first, primarily because of that phone call he got two years ago. “It’s a project that’s really close to my heart,” he says. “Giora’s not a family member or anything, but just knowing I have the skills that could help someone like him, it’s become a calling.”