Unleashing the Potential of AI: Using Video Games and Virtual Dogs to Create 3D Models

Wednesday - March 6 2024, 15:38 UTC - 8 months ago

The University of Surrey has found a way to use images from Grand Theft Auto V to train an AI system to accurately predict 3D poses of dogs, eliminating the need for costly motion capture equipment. The resulting database, DigiDogs, has potential applications in conservation, animation, and virtual gaming. The research won Best Paper at a prestigious computer vision conference, highlighting the potential of this innovative approach.

Photographs of dogs have long been used as a reference for artists and animal lovers alike, but what if these images could serve a larger purpose? A recent study by the University of Surrey has found a way to use images of dogs to generate highly accurate 3D models—thanks to an unexpected collaborator: the popular video game Grand Theft Auto V.

The team of researchers, led by Postgraduate Research Student Moira Shooter, trained an artificial intelligence (AI) system to predict the 3D pose of a dog using only a 2D image. To do this, they turned to Grand Theft Auto V, a game known for its realistic graphics and dynamic environments. By altering the game's code and replacing the main character with one of eight different dog breeds, the team was able to generate a database of 27,900 frames featuring dogs in various movements and weather conditions.

Traditionally, teaching AI systems to gather 3D information from 2D images requires the use of motion capture suits for human subjects. However, the team's innovative approach eliminated the need for such equipment, making the process more efficient and cost-effective. The result was a database, aptly called DigiDogs, which was used to train the AI model with Meta's DINOv2 technology to accurately predict 3D poses.

But why use virtual dogs instead of real ones? Shooter explains, "our model was trained on CGI dogs, but we were able to use it to make 3D skeletal models from photographs of real animals. That could let conservationists spot injured wildlife, or help artists create more realistic animals in the metaverse." The potential applications of this technology are vast, from aiding in conservation efforts to enhancing the virtual gaming experience.

The success of the study did not go unnoticed. The research paper won the prize for Best Paper at the IEEE/CVF's Winter Conference on Applications of Computer Vision, solidifying the potential of this approach in various fields. "3D poses contain so much more information than 2D photographs," says Shooter, "from ecology to animation—this neat solution has so many possible uses." .

As AI technology continues to advance, the possibilities for its application are endless. The University of Surrey's collaboration with Grand Theft Auto and the use of virtual dogs to generate 3D models is just one example of the potential of this technology. With further development and fine-tuning, this method could revolutionize multiple industries and change the way we interact with the world around us.