Generative AI in the Workplace - Exploring Its Impact on Performance and Productivity

Saturday - April 29 2023, 16:49 UTC - 1 year ago

A study done by scholars from the Stanford Digital Economy Lab and the Massachusetts Institute of Technology into the impact of generative AI deployed at a call center found access to AI assistance improved worker productivity by 14%, with the biggest impact on less experienced workers. Few studies have examined the impact on productivity, organizational structure, or morale of AI at workplace. Results of the study show that high-skilled workers may have less to gain from AI assistance because AI recommendations capture the knowledge embodied in their own behaviors.

As generative AI increases in capability, these models are moving from the lab to the office. But do they improve worker productivity? .

Scholars from the Stanford Digital Economy Lab within Stanford HAI and the Massachusetts Institute of Technology studied the impact of generative AI deployed at scale in the customer service sector at a call center. They found that access to AI assistance increased agent productivity by 14%, with the biggest impact on less experienced workers.

"We found that workers with access to AI see fairly significant productivity gains, but most of those gains accrue to novice or less able workers. This may be because the AI model disseminates the potentially tacit knowledge of more able workers and helps new workers move up the experience curve," says Lindsey Raymond, an MIT Ph.D. candidate and co-author of the new working paper titled "Generative AI at Work." .

The research team—which includes economist Erik Brynjolfsson, director of the Stanford Digital Economy Lab; Danielle Li, associate professor at the MIT Sloan School of Management; and Raymond—examined the staggered deployment of a chat assistant for a Fortune 500 software firm that provides business process software. The tool, trained on data from over 5,000 agents at the company, monitors customer chats and offers company agents real-time suggestions for how to respond to customers. The agents could use those suggestions but were also free to ignore them.

The scholars found that the tool allowed agents to resolve 13.8% more issues per hour: Agents were able to move through issues more quickly, handle multiple calls at once, and were more successful at resolving issues. The AI assistant improved the performance of less skilled or less experienced workers across all productivity measures. Agents with two months of tenure who used the tool were able to perform as well as agents with six months of tenure who didn't have access to the AI.

The researchers found few positive effects of AI for the highest skilled or most experienced members of the company. "High-skilled workers may have less to gain from AI assistance precisely because AI recommendations capture the knowledge embodied in their own behaviors," says Brynjolfsson.

Outside of worker productivity, Brynjolfsson noted that the AI assistance had an additional benefit: As a result of the AI suggestions that were designed to help the agents be more empathetic with frustrated customers, customers treated the agents far more positively.

As generative AI and foundation models more broadly improve performance, more of these tools are finding their way into the workforce. But thus far, few studies have examined in any large scale their impact on productivity, organizational structure, or morale, the authors note. A small but growing number of studies have tried to understand generative AI capabilities, including some small-scale experiments, but this is one of the first studies to look at generative AI tools in the real-world workplace.

"We need far more research here," says Brynjolfsson. "We don't know if the impact of AI on productivity may vary over time, and adding these tools to the office could require complementary organizational investments, skills development, and other changes like different job titles. So there is still so much we don't know." .