Study on AI Tapping Into Actors' Emotions

Saturday - October 21 2023, 05:55 UTC - 1 year ago

In September 2023, a Hollywood studio hosted a emotion AI study project that trained AI to express human emotions using recordings of actors' faces, voices and movements. The project has sparked debate on the actors' rights in the age of AI as questions regarding the commercial use of their likenesses and informed consent arise.

One evening in early September, T, a 28-year-old actor who asked to be identified by his first initial, took his seat in a rented Hollywood studio space in front of three cameras, a director, and a producer for a somewhat unusual gig. The two-hour shoot produced footage that was not meant to be viewed by the public—at least, not a human public. Rather, T’s voice, face, movements, and expressions would be fed into an AI database "to better understand and express human emotions .

" That database would then help train "virtual avatars" for Meta, as well as algorithms for a London-based emotion AI company called Realeyes. (Realeyes was running the project; participants only learned about Meta’s involvement once they arrived on site.) The "emotion study" ran from July through September, specifically recruiting actors. The project coincided with Hollywood’s historic dual strikes by the Writers Guild of America and the Screen Actors Guild (SAG-AFTRA) .

With the industry at a standstill, the larger-than-usual number of out-of-work actors may have been a boon for Meta and Realeyes: here was a new pool of "trainers"—and data points—perfectly suited to teaching their AI to appear more human.For actors like T, it was a great opportunity too: a way to make good, easy money on the side, without having to cross the picket line."This is fully a research-based project," the job posting said .

It offered $150 per hour for at least two hours of work, and asserted that "your individual likeness will not be used for any commercial purposes."The actors may have assumed this meant that their faces and performances wouldn’t turn up in a TV show or movie, but the broad nature of what they signed makes it impossible to know the full implications for sure. In fact, in order to participate, they had to sign away certain rights "in perpetuity" for technologies and use cases that may not yet exist .

And while the job posting insisted that the project "does not qualify as struck work" (that is, work produced by employers against whom the union is striking), it nevertheless speaks to some of the strike’s core issues: how actors’ likenesses can be used, how actors should be compensated for that use, and what informed consent should look like in the age of AI."This isn’t a contract battle between a union and a company," said Duncan Crabtree-Ireland, SAG-AFTRA’s chief negotiator, at a panel on AI in entertainment at San Diego Comic-Con this summer .

"It’s existential.Many actors across the industry, particularly background actors (also known as extras), worry that AI—much like the models described in the emotion study—could be used to replace them, whether or not their exact faces are copied. And in this case, by providing the facial expressions that will teach AI to appear more human, study participants may in fact have been the ones inadvertently training their own potential replacements .

"Our studies have nothing to do with the strike," Max Kalehoff, Realeyes’s vice president for growth and marketing, said in an email. "The vast majority of our work is in evaluating the effectiveness of advertising for clients—which has nothing to do with creating perfect AI 'actors.'" .