ChatGPT Could Be Used by Less Experienced Workers to Produce Quality Writing

Tuesday - July 18 2023, 08:28 UTC - 1 year ago

A study by two MIT economics students suggests that using ChatGPT could help reduce gaps in writing ability between employees. The research showed that those who used ChatGPT produced higher-quality work in less time, while those who did not use the chatbot faced possible serious implications. AI could be helpful in the workplace by acting as a sort of virtual assistant.

People have been using ChatGPT to help them to do their jobs since it was released in November of last year, with enthusiastic adopters using it to help them write everything from marketing materials to emails to reports. Now we have the first indication of its effect in the workplace. A new study by two MIT economics graduate students, published today in Science, suggests it could help reduce gaps in writing ability between employees. They found that it could enable less experienced workers who lack writing skills to produce work similar in quality to that of more skilled colleagues.

Shakked Noy and Whitney Zhang recruited 453 marketers, data analysts, and college-educated professionals and got each of them to complete two kinds of tasks they’d normally undertake as part of their jobs, such as writing press releases, short reports, or analysis plans. Half were given the option of using ChatGPT to help them complete the second of the two tasks.

A group of other professionals then quality-checked the results, grading the writing on a scale of 1 to 7, with 7 the best. Each piece of work was evaluated by three people working in the same professions, hired through the research platform Prolific.The writers who chose to use ChatGPT took 40% less time to complete their tasks, and produced work that the assessors scored 18% higher in quality than that of the participants who didn’t use it. The writers who were already skilled at writing were able to reduce the amount of time they spent on their work, while those who were assessed as being weaker writers produced higher-quality work once they gained access to the chatbot.

"ChatGPT is just very good at producing this kind of written content, and so using it to automate parts of the writing process seems likely to save a lot of time," says Noy, lead author of the research."One thing that’s clear is that this is very useful for white-collar work—a lot of people will be using it, and it’s going to have a pretty big effect on how white-collar work is structured," he adds.

Depending on the nature of a person’s job, those kinds of inaccuracies could have serious implications. Lawyer Steven Schwartz was fined $5,000 by a judge last month for using ChatGPT to produce a legal brief that contained false judicial opinions and legal citations."Technological advances are commonplace and there is nothing inherently improper about using a reliable artificial intelligence tool for assistance," the judge, Kevin Castel, wrote. "But existing rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings." .

The research hints at how AI could be helpful in the workplace by acting as a sort of virtual assistant, says Riku Arakawa, a researcher at Carnegie Mellon University who studies workers’ use of large language models, and was not involved with the research. "I think this is a really interesting result that demonstrates how human-AI cooperation works really well in this kind of task. When a human leverages AI to refine their output, they can produce better content," he adds.