Google DeepMind's Levels of AGI Framework

Thursday - December 7 2023, 08:16 UTC - 11 months ago

Google DeepMind is attempting to make the discussion of Artificial General Intelligence (AGI) more precise and tangible by proposing a framework that outlines different levels of AGI, with the latest chatbots representing AGI level 1. Their framework focuses on capabilities, thresholds of performance, generality, embodiment, ecological validity, and charting progress rather than reaching a single endpoint. With this framework, AGI level 1 chatbots are able to understand natural language and output responses in an informed way.

Artificial general intelligence, or AGI, has become a much-abused buzzword in the AI industry. Now, Google DeepMind wants to put the idea on a firmer footing. The concept at the heart of the term AGI is that a hallmark of human intelligence is its generality. While specialist computer programs might easily outperform us at picking stocks or translating French to German, our superpower is the fact we can learn to do both. Recreating this kind of flexibility in machines is the holy grail for many AI researchers, and is often speculated to be the first step towards artificial superintelligence. But what exactly people mean by AGI is rarely specified, and the idea is frequently described in binary terms, where AGI represents a piece of software that has crossed some mythical boundary, and once on the other side, it’s on par with humans.

Researchers at Google DeepMind are now attempting to make the discussion more precise by concretely defining the term. Crucially, they suggest that rather than approaching AGI as an end goal, we should instead think about different levels of AGI, with today’s leading chatbots representing the first rung on the ladder. "We argue that it is critical for the AI research community to explicitly reflect on what we mean by AGI, and aspire to quantify attributes like the performance, generality, and autonomy of AI systems," the team writes in a preprint published on arXiv.

The researchers note that they took inspiration from autonomous driving, where capabilities are split into six levels of autonomy, which they say enable clear discussion of progress in the field. To work out what they should include in their own framework, they studied some of the leading definitions of AGI proposed by others. By looking at some of the core ideas shared across these definitions, they identified six principles any definition of AGI needs to conform with.

For a start, a definition should focus on capabilities rather than the specific mechanisms AI uses to achieve them. This removes the need for AI to think like a human or be conscious to qualify as AGI. They also suggest that generality alone is not enough for AGI, the models also need to hit certain thresholds of performance in the tasks they perform. This performance doesn’t need to be proven in the real world, they say—it’s enough to simply demonstrate a model has the potential to outperform humans at a task.

While some believe true AGI will not be possible unless AI is embodied in physical robotic machinery, the DeepMind team say this is not a prerequisite for AGI. The focus, they say, should be on tasks that fall in the cognitive and metacognitive—for instance, learning to learn—realms.

Another requirement is that benchmarks for progress have "ecological validity," which means AI is measured on real-world tasks valued by humans. And finally, the researchers say the focus should be on charting progress in the development of AGI rather than fixating on a single endpoint. Based on these principles, the team proposes a framework they call "Levels of AGI" that outlines a way to categorize algorithms based on their performance and generality.

An AGI level 1 chatbot, for instance, could have a basic understanding of natural language, and use this to respond in an informed way. This AI is not yet able to solve the kinds of challenging tasks machines can handle today—like object recognition or translation. But it has the potential to apply its core skills to more demanding tasks. At later levels, the AI can achieve provable benchmarking in areas like scientific discovery, visual reasoning, and navigation.

The DeepMind team cautions that defining AGI is still a work in progress, and say this new framework is only meant to provide a way for researchers to discuss progress in the field, as well as suggest directions for research. When it comes to deciphering AGI, there’s still much more work to be done, they say.