Can Detectors Spot Text Generated by AI?

Saturday - July 8 2023, 12:08 UTC - 1 year ago

Within weeks of ChatGPT's launch, there were fears that students would use it to spin up essays in seconds. To combat this, detection tools were created to spot text written by a machine. The study found all the tested tools had difficulty seeing AI-generated text that was slightly tweaked and edited by humans, suggesting students can easily adapt the AI-generated texts for detection. This also highlighted how outdated universities' current methods are, with the potential to detrimentally affect students and teachers if set up incorrectly.

Within weeks of ChatGPT’s launch, there were fears that students would be using the chatbot to spin up passable essays in seconds. In response to those fears, startups started making products that promise to spot whether text was written by a human or a machine.

The problem is that it’s relatively simple to trick these tools and avoid detection, according to new research that has not yet been peer reviewed.

Debora Weber-Wulff, a professor of media and computing at the University of Applied Sciences, HTW Berlin, worked with a group of researchers from a variety of universities to assess the ability of 14 tools, including Turnitin, GPT Zero, and Compilatio, to detect text written by OpenAI’s ChatGPT.

Most of these tools work by looking for hallmarks of AI-generated text, including repetition, and then calculating the likelihood that the text was generated by AI. But the team found that all those tested struggled to pick up ChatGPT-generated text that had been slightly rearranged by humans and obfuscated by a paraphrasing tool, suggesting that all students need to do is slightly adapt the essays the AI generates to get past the detectors. "These tools don’t work," says Weber-Wulff. "They don’t do what they say they do. They’re not detectors of AI." .

The researchers assessed the tools by writing short undergraduate-level essays on a variety of subjects, including civil engineering, computer science, economics, history, linguistics, and literature. They wrote the essays themselves to be certain the text wasn’t already online, which would have meant it might already have been used to train ChatGPT. Then each researcher wrote an additional text in Bosnian, Czech, German, Latvian, Slovak, Spanish, or Swedish. Those texts were passed through either the AI translation tool DeepL or Google Translate to translate them into English.

The team then used ChatGPT to generate two additional texts each, which they slightly tweaked in an effort to hide that it’d been AI-generated. One set was edited manually by the researchers, who reordered sentences and exchanged words, while another was rewritten using an AI paraphrasing tool called Quillbot. In the end, they had 54 documents to test the detection tools on.

They found that while the tools were good at identifying text written by a human (with 96% accuracy, on average), they fared more poorly when it came to spotting AI-generated text, especially when it had been edited. Although the tools identified ChatGPT text with 74% accuracy, this fell to 42% when the ChatGPT-generated text had been tweaked slightly.

These kinds of studies also highlight how outdated universities’ current methods for assessing student work are, says Vitomir Kovanović, a senior lecturer who builds machine-learning and AI models at the University of South Australia, who was not involved in the project.

Daphne Ippolito, a senior research scientist at Google specializing in natural-language generation, who also did not work on the project, raises another concern. "If automatic detection systems are to be employed in education settings, it is crucial to understand their rates of false positives, as incorreclty flagged documents can detrimentally affect students and educators," says Ippolito.