The Possibility of Using ChatGPT and Other Large Language Models to Probe Into Societal Mind

Tuesday - August 1 2023, 14:23 UTC - 1 year ago

Thanks to the advent of ChatGPT and other large language models (LLMs), social scientists are considering using these tools to rapidly construct diverse groups of 'simulated humans' and run experiments to probe their behavior and values as a proxy to their biological counterparts. AI could change the game for social science research, allowing researchers to collect data and test theories about human behavior with unprecedented speed. Social scientists use a variety of methods, such as questionnaires, to test their hypothesis. LLMs can assist in uncovering biases in data, discerning patterns of behaviors related to emotions and political opinion, and can understand languages like irony and sarcasm.

I’m a huge fan of Anthony Bourdain’s travel show Parts Unknown. In each episode, the chef visits remote villages across the globe, documenting the lives, foods, and cultures of regional tribes with an open heart and mind. The show provides a glimpse into humanity’s astonishing diversity. Social scientists have a similar goal—understanding the behavior of different people, groups, and cultures—but use a variety of methods in controlled situations. For both, the stars of these pursuits are the subjects: humans.

But what if you replaced humans with AI chatbots? .

The idea sounds preposterous. Yet thanks to the advent of ChatGPT and other large language models (LLMs), social scientists are flirting with the idea of using these tools to rapidly construct diverse groups of "simulated humans" and run experiments to probe their behavior and values as a proxy to their biological counterparts.

If you’re imagining digitally recreated human minds, that’s not it. The idea is to tap into ChatGPT’s expertise at mimicking human responses. Because the models scrape enormous amounts of online data—blogs, Youtube comments, fan fiction, books—they readily capture relationships between words in multiple languages. These sophisticated algorithms can also decode nuanced aspects of language, such as irony, sarcasm, metaphors, and emotional tones, a critical aspect of human communication in every culture. These strengths set LLMs up to mimic multiple synthetic personalities with a wide range of beliefs.

Another bonus? Compared to human participants, ChatGPT and other LLMs don’t get tired, allowing scientists to collect data and test theories about human behavior with unprecedented speed.

The idea, though controversial, already has support. A recent article reviewing the nascent field found that in certain carefully-designed scenarios, ChatGPT’s responses correlated with those of roughly 95 percent of human participants.

AI "could change the game for social science research," said Dr. Igor Grossman at the University of Waterloo, who with colleagues recently penned a look-ahead article in Science. The key for using Homo silicus in research? Careful bias management and data fidelity, said the team.

Probing the Human Societal Mind .

What exactly is social science? Put simply, it’s studying how humans—either as individuals or as a group—behave under different circumstances, how they interact with each other and develop as a culture. It’s an umbrella of academic pursuit with multiple branches: economics, political science, anthropology, and psychology. The discipline tackles a wide range of topics prominent in the current zeitgeist. What’s the impact of social media on mental health? What are current public attitudes towards climate change as severe weather episodes increase? How do different cultures value methods of communication—and what triggers misunderstandings? .

A social science study starts with a question and a hypothesis. One of my favorites: do cultures tolerate body odor differently? (No kidding, the topic has been studied quite a bit, and yes, there is a difference!) Scientists then use a variety of methods, like questionnaire surveys, to test the hypothesis. With the tantalizing promise of artificial intelligence, it’s possible that one day scientists will use ChatGPT and other LLMs to probe questions about the human condition in record time.