Summary: A team of researchers predicts that artificial intelligence (AI), especially large language models (LLMs), could redefine research in the social sciences.
They believe LLMs, trained on large amounts of text data, can mimic human responses to aid in large and rapid studies of human behavior. Traditional data collection methods in the social sciences could see a significant shift as a result of these advances.
However, the researchers warn of potential pitfalls, such as AI’s inability to replicate socio-cultural biases and the need for transparent, open-source AI models to ensure research equity and quality.
Main aspects:
- LLMs could potentially replace human participants for data collection, as they have already demonstrated the ability to generate realistic survey responses in fields such as consumer behaviour.
- The use of artificial intelligence in the social sciences opens up new ways to generate hypotheses that can later be confirmed in human populations.
- While LLMs offer vast potential, they often exclude socio-cultural biases that exist in real human populations, presenting a significant challenge to researchers studying these biases.
Source: University of Waterloo
In an article published yesterday in the prestigious magazineScienceLeading researchers from the University of Waterloo, the University of Toronto, Yale University and the University of Pennsylvania examine how artificial intelligence (Large Language Models or LLMs in particular) could change the nature of their work .
What we wanted to explore in this article is how research practices in the social sciences can be adapted, even reinvented, to harness the power of artificial intelligence, said Igor Grossmann, professor of psychology at Waterloo.
Grossmann and colleagues note that large language models trained on large amounts of text data are increasingly able to simulate human-like responses and behaviors. This offers new opportunities to test theories and hypotheses about human behavior at scale and speed.
Traditionally, the social sciences have relied on a variety of methods, including questionnaires, behavioral tests, observational studies, and experiments.
A common goal in social science research is to obtain a generalized representation of the characteristics of individuals, groups, cultures and their dynamics. With the advent of advanced AI systems, the landscape of data collection in the social sciences may change.
AI models can represent a diverse range of human experiences and perspectives, perhaps giving them a higher degree of freedom to generate different responses than conventional methods of human participants, which may help reduce generalizability issues in research, Grossman said.
LLMs could displace human participants for data collection, said Philip Tetlock, a psychology professor at UPenn.
Indeed, LLMs have already demonstrated their ability to generate realistic responses to consumer behavior surveys. Big language models will revolutionize human-based predictions in the next 3 years.
“It makes no sense for unaided humans to make probabilistic judgments in serious political debates. I put a 90% chance on it. Of course, how humans react to all of this is another matter.
While opinions about the feasibility of this application of advanced AI systems vary, studies using simulated participants could be used to generate new hypotheses that could then be confirmed in human populations.
But the researchers warn of some of the possible pitfalls in this approach, including the fact that LLMs are often trained to rule out socio-cultural biases that exist for real-life humans. This means that sociologists using AI in this way couldn’t study those biases.
University of Waterloo co-author Professor Dawn Parker notes that researchers will need to establish guidelines for the governance of LLMs in research.
Pragmatic concerns about data quality, fairness, and fairness of access to powerful AI systems will be substantial, Parker said.
Hence, we need to ensure that LLMs in social sciences, like all scientific models, are open-source, meaning that their algorithms and ideally data are available for all to scrutinize, test and modify.
“Only by maintaining transparency and replicability can we ensure that AI-assisted social science research truly contributes to our understanding of human experience.
Learn about this AI research news story
Author: Ryon Jones
Source: University of Waterloo
Contact: Ryon Jones – University of Waterloo
Image: The image is credited to Neuroscience News
Original research: Free access.
“AI and the Transformation of Social Science Research” by Igor Grossmann et al. Science
Abstract
AI and the transformation of research in the social sciences
Advances in artificial intelligence (AI), particularly large language models (LLMs), are substantially influencing research in the social sciences.
These transformer-based machine learning models pretrained on large amounts of text data are increasingly capable of simulating human-like responses and behaviors, providing opportunities to test theories and hypotheses about human behavior at scale and speed.
This presents pressing challenges: How can social science research practices be adapted, even reinvented, to harness the power of fundamental AI? And how can this be done while ensuring transparent and replicable research?
#Revolution #Simulating #Human #Behavior #Breakthrough #Social #Sciences #Neuroscience #News