AI Image-Generators Found To Include Images of Child Sexual Abuse

Thursday - December 28 2023, 05:03 UTC - 11 months ago

A new report found more than 3,200 images of suspected child sexual abuse in a popular AI database, which has caused companies to take action and development of AI technologies to be more closely reviewed. It is believed that AI tools are being used by pedophiles to generate realistic and explicit imagery of fake children as well as transform social media photos of fully clothed real teens into nudes.

Hidden inside the foundation of popular artificial intelligence image-generators are thousands of images of child sexual abuse, according to a new report that urges companies to take action to address a harmful flaw in the technology they built. Those same images have made it easier for AI systems to produce realistic and explicit imagery of fake children as well as transform social media photos of fully clothed real teens into nudes, much to the alarm of schools and law enforcement around the world .

Until recently, anti-abuse researchers thought the only way that some unchecked AI tools produced abusive imagery of children was by essentially combining what they've learned from two separate buckets of online images—adult pornography and benign photos of kids. But the Stanford Internet Observatory found more than 3,200 images of suspected child sexual abuse in the giant AI database LAION, an index of online images and captions that's been used to train leading AI image-makers such as Stable Diffusion .

The watchdog group based at Stanford University worked with the Canadian Centre for Child Protection and other anti-abuse charities to identify the illegal material and report the original photo links to law enforcement. It said roughly 1,000 of the images it found were externally validated. The response was immediate. On the eve of the Wednesday release of the Stanford Internet Observatory's report, LAION told The Associated Press it was temporarily removing its datasets .

LAION, which stands for the nonprofit Large-scale Artificial Intelligence Open Network, said in a statement that it "has a zero tolerance policy for illegal content and in an abundance of caution, we have taken down the LAION datasets to ensure they are safe before republishing them.". While the images account for just a fraction of LAION's index of some 5.8 billion images, the Stanford group says it is likely influencing the ability of AI tools to generate harmful outputs and reinforcing the prior abuse of real victims who appear multiple times .

It's not an easy problem to fix, and traces back to many generative AI projects being "effectively rushed to market" and made widely accessible because the field is so competitive, said Stanford Internet Observatory's chief technologist David Thiel, who authored the report. "Taking an entire internet-wide scrape and making that dataset to train models is something that should have been confined to a research operation, if anything, and is not something that should have been open-sourced without a lot more rigorous attention," Thiel said in an interview .

A prominent LAION user that helped shape the dataset's development is London-based startup Stability AI, maker of the Stable Diffusion text-to-image models. New versions of Stable Diffusion have made it much harder to create harmful content, but an older version introduced last year—which Stability AI says it didn't release—is still baked into other applications and tools and remains "the most popular model for generating explicit imagery," according to the Stanford report .

"We can't take that back. That model is in the hands of many people on the internet," Thiel said.