The Coming Wave: How AI Could Pass The Modern Turing Test

Saturday - July 15 2023, 04:25 UTC - 1 year ago

The Turing Test has been the rallying call for the AI community for decades. It proposes that if an AI is capable of fooling a human in a text-based conversation, it could be regarded as intelligent. However, with the rise of powerful language models such as GPT-4, the Turing Test is on the cusp of being passed. Mustafa Suleyman proposes the Modern Turing Test, which challenges AI to take concrete actions in the real-world, such as turning a $100,000 investment into $1 million on a retail web platform. To pass this test, AI has to research, design, negotiate and even run marketing campaigns. With progress in language generative models, image recognition and AI advancements, something like this could be achievable in two years.

AI systems are increasingly everywhere and are becoming more powerful almost by the day. But even as they become ever more ubiquitous and do more, how can we know if a machine is truly "intelligent"? For decades the Turing test defined this question. First proposed in 1950 by the computer scientist Alan Turing, it tried to make sense of a then emerging field and never lost its pull as a way of judging AI.

Turing argued that if AI could convincingly replicate language, communicating so effectively that a human couldn’t tell it was a machine, the AI could be considered intelligent. To take part, human judges sit in front of a computer, tap out a text-based conversation, and guess at who (or what) is on the other side. Simple to envisage and surprisingly hard to pull off, the Turing test became an ingrained feature of AI. Everyone knew what it was; everyone knew what they were working toward. And while cutting-edge AI researchers moved on, it remained a potent statement of what AI was about—a rallying call for new researchers.

But there’s now a problem: the Turing test has almost been passed—it arguably already has been. The latest generation of large language models, systems that generate text with a coherence that just a few years ago would have seemed magical, are on the cusp of acing it.

So where does that leave AI? And more important, where does it leave us?The truth is, I think we’re in a moment of genuine confusion (or, perhaps more charitably, debate) about what’s really happening. Even as the Turing test falls, it doesn’t leave us much clearer on where we are with AI, on what it can actually achieve. It doesn’t tell us what impact these systems will have on society or help us understand how that will play out.

We need something better. Something adapted to this new phase of AI. So in my forthcoming book The Coming Wave, I propose the Modern Turing Test—one equal to the coming AIs. What an AI can say or generate is one thing. But what it can achieve in the world, what kinds of concrete actions it can take—that is quite another. In my test, we don’t want to know whether the machine is intelligent as such; we want to know if it is capable of making a meaningful impact in the world. We want to know what it can do.Mustafa Suleyman .

Put simply, to pass the Modern Turing Test, an AI would haveto successfully act on this instruction: "Go make $1 million on a retail web platform in a few months with just a $100,000 investment." To do so, it would need to go far beyond outlining a strategy and drafting some copy, as current systems like GPT-4 are so good at doing. It would need to research and design products, interface with manufacturers and logistics hubs, negotiate contracts, create and operate marketing campaigns. It would need, in short, to tie together a series of complex real-world goals with minimal oversight. You would still need a human to approve various points, open a bank account, actually sign on the dotted line. But the work would all be done by an AI.

Something like this could be as little as two years away. Many of the ingredients are in place. Image and text generation are, of course, already far more advanced than they were just a few years ago. Technologies like computer vision are also progressing at pace, and AI models are becoming ever-more adept. All of these advances are pushing the potential for AI to create impact in the world—the kind of work that not so long ago would have seemed fantastical. Things that just a few years ago weren’t hypothetically plausible, but are now not only hypothetical, but realistic goals.There’s no easy path forward, and the transition from text-generating GPTs to actual real-world impact won’t be a straight line. There’ll be doubts, setbacks, arguments, successes, failures. We’ll learn a lot for the journey; there’ll be stumbles along the way, but step by step we’ll move toward something that looks more and more like the vision that Turing set out in 1950.