Kevin Hu makes faces in his GIF mirror at the MIT Media Lab.
Jon Christian
Deep in the bowels of the avant-garde, glass and metal MIT Media Lab, graduate student Kevin Hu is making faces into an ornate mirror.
He opens his eyes and mouth as wide as possible in a caricature of shock. A hidden webcam analyzes his facial expression in real time, digs through a vast database for GIFs that convey a similar emotion, and projects them on the surface of the mirror, against Hu’s reflection. In quick succession it spits out a series of disparate images: a surprised anime character, an affronted Walter White, and then a man in a crowd with an astonished, wide-open mouth much like Hu’s own.
Next Hu contorts his face into a rictus-like grin (“I can smile,” he mutters) and an exuberant basketball player appears on the mirror before being replaced by Snow White, who claps her hands in delight. She’s not emulating Hu’s face exactly, but when it comes to finding a GIF for every mood, she’s a fairly decent simulacrum.
Hu and collaborator Travis Rich, a PhD candidate at the Media Lab, built the mirror to demonstrate a remarkable ongoing project meant to find a whole new use for one of the Internet’s favorite toys. Back in March, the two launched a site called GIFGIF, which had a modest premise: Show people a pair of random GIFs, and ask them which better expresses a given emotion. For instance, it might ask you whether Arrested Development’s Lucille Bluth or a gloomy Kurt Cobain seems more surprised. Or it might show you a bowing Robin Hood from Disney’s 1973 animated feature and a shrugging Donald Glover, and ask which better expresses pleasure. Sometimes the answer is clear; if it isn’t, you can click “neither.”
The goal was to harness crowdsourcing to map emotions, a task at which computers are very poorly equipped. Eventually, Hu and Rich hope, all that subjective data will make it easier to write programs that deal with emotional content.
“There are all these things that have meaning to us,” says Rich. “But it’s hard to translate those into code.”
Giving Programmers Tools to Help Machines Understand Feelings
After its launch, GIFGIF quickly went viral—helped along by mentions in, among others, USA Today and The Washington Post —and the corresponding explosion in traffic jumpstarted a database that has since grown to include more than 2.7 million votes. That trove of GIFs, each tagged with weighted emotional characteristics, opens up some unprecedented possibilities. For example, you can query it for a GIF that’s 60 percent amused, 30 percent disgusted, and 10 percent relieved, with results that often show startling insight. These capabilities make it a potential goldmine for everyone from researchers who study facial expressions to app developers who want to suggest content based on a user’s emotional needs.
It’s with those sorts of applications in mind that Hu and Rich are now preparing to release two tools that build on GIFGIF. The first, an open API being released this week, will let anyone with an app or website query the dataset to return a GIF with particular emotional content. It’s already opened up new avenues for researchers. “Travis and Kevin are doing some awesome work,” says Brendan Jou, a PhD candidate at Columbia University who recently published a paper on predicting perceived emotions using an alpha version of the GIFGIF API.
But it’s the tool that’s coming after the API, a platform they call Quantify that they’ll be releasing later this month, which opens up even deeper possibilities.
The idea behind Quantify is to let anybody start a project like GIFGIF, including for things other than GIFs. A project about food, for example, could build a dataset of which meals or dishes respondents see as appropriate for specific contexts and slowly build an index of food concepts for various scenarios. For example, you probably wouldn’t eat mashed potatoes and gravy on a warm summer morning, but you likely crave ice cream when you’re sad or want home-cooked dinners when you’re lonely. With enough responses in a campaign about food, a programmer could write an app that recommends grub based on your emotional state. It could even glean respondents’ relative locations using IP addresses—information that can be used to determine if those recommendations should be different based on the user’s region.
Broader Applications
Quantify also presents tantalizing possibilities for marketers. An automobile manufacturer, say, could create a project that showed conceptual dashboards or steering wheels to respondents in order to develop data on what consumers associate with nebulous concepts like safety or luxury. Though they won’t divulge who, Hu and Rich say they’ve already had discussions about Quantify with several high profile corporate sponsors at the Media Lab.
“Now, instead of having a designer that knows all of these things, you can sort of programmatically say, ‘OK, it’s for a Chinese market, and they prefer this mixture of luxury and safety so we’ll design it this way,'” Rich says. “Because we have all this human data that’s being collected and IP located, we know what German preferences are and what Chinese preferences are and what Brazilian preferences are.”
There are also broad applications in the social sciences. To test Quantify, Hu and Rich helped Carnegie Mellon professor William Alba develop a project called Earth Tapestry, which shows pairs of locations (Mount Kilimanjaro, the Large Hadron Collider, Stonehenge) and asks which better expresses various properties (durability, nobility, delightfulness). If all goes according to plan, the dataset collected on Earth Tapestry will be laser-engraved on a sapphire disk and sent to the Moon on the Astrobotic lunar lander by 2016.
“I wrote Travis and Kevin last May because I had been seeking a method that would translate individual pairwise choices into a ranking,” Alba says. “They went light-years further than I had hoped.”
And that’s just a taste of what they’ve tried so far. Rich and Hu say being able to teach computers how to recommend based on feelings and emotions could have applications in fields from psychological and behavioral studies to artificial intelligence. It just depends on how programmers want to use them. One app Rich says he’d love to see is one that analyzes the text of an instant message and suggests a GIF that matches its emotional palette. (No more searching “Beyoncé side-eye” when your friend tells you about a bad date!)
Back in the Media Lab, Hu again steps in front of the mirror and tries an even more exaggerated look of astonishment. The mirror goes blank for a moment, then it loops a GIF of a wild-eyed skydiver waving his arms in free-fall.
“That’s a good surprised one,” Rich says to Hu. “Were you trying to be surprised?”
No comments:
Post a Comment