Artificial intelligence-powered writing tools such as autocomplete suggestions can definitely change the way people express themselves, but can they also change how they think? Cornell Tech researchers think so.
In two large-scale experiments, participants were exposed to a biased AI writing assistant that provided autocomplete suggestions as they wrote about societal issues like whether the death penalty should be abolished or whether fracking should be allowed. Using pre- and post-experiment surveys, the researchers found that participants who used the biased AI had their views gravitate toward the AI’s positions.
What’s more, participants were unaware of the shifts in their opinions – and explaining the AI’s bias to the participants, either before or after the exercise, didn’t mitigate AI’s influence.
“Previous misinformation research has shown that warning people before they’re exposed to misinformation, or debriefing them afterward, can provide ‘immunity’ against believing it,” said Sterling Williams-Ceci ’21, a doctoral candidate in information science. “So we were surprised because neither of those interventions actually reduced the extent to which people’s attitudes shifted toward the AI’s bias in this context.”
Williams-Ceci is the lead author of “Biased AI Writing Assistants Shift Users’ Attitudes on Societal Issues,” published March 11 in Science Advances. This work extends a project started by co-author Maurice Jakesch, Ph.D. ’24, now assistant professor of computer science at Bauhaus University in Weimar, Germany.
Senior author Mor Naaman, the Don and Mibs Follett Professor of Information Science at Cornell Tech, the Jacobs Technion-Cornell Institute at Cornell Tech and the Cornell Ann S. Bowers College of Computing and Information Science, said a couple of things have happened that made extending the group’s previous research important.
“For one, autocomplete is everywhere now,” Naaman said. “It was less prevalent and limited to short completions three years ago, but these days Gmail, for example, will suggest writing entire emails on your behalf. Second, when we first wrote the paper, people were saying, ‘Why would AI be purposefully biased?’ But since then, it has become clear that bias explicitly built into AI interactions is a very plausible scenario.”
Naaman and the group also found in the latest work that biased AI suggestions have the power to shift attitudes “across different topics, and across different political leanings.”
In the two studies, together involving more than 2,500 people, the group found consistently that participants’ attitudes shifted toward the biased AI suggestions. In one study, participants were asked to write a short essay for or against standardized testing being used in education. Participants either saw biased autocomplete suggestions favoring testing or did not; a third group, instead of auto-complete suggestions, was shown a list of pro-testing arguments, generated by the AI prior to the experiment, and these participants’ attitudes did not shift as much.
The second experiment broadened the scope, asking participants to write about politically consequential topics including the death penalty, fracking, genetically modified organisms and voting rights for felons.
For each issue, the researchers engineered AI suggestions to gravitate toward a predetermined bias; opinions were liberal-leaning for death penalty and GMOs, conservative-leaning for felons’ voting and fracking. Additionally, some participants were made aware of the bias in the AI, either before or after writing.
In every experiment, the researchers found that participants’ views shifted in the direction of the AI bias. The biggest surprise, Naaman said, was that mitigation measures did not work.
“We told people before, and after, to be careful, that the AI is going to be (or was) biased, and nothing helped,” Naaman said. “Their attitudes about the issues still shifted.”
It’s well understood that people’s attitudes influence their behaviors, and even that people’s behavior shifts their attitudes, said Williams-Ceci. But here, the influence is covert: People do not notice it, and are unable to resist it, she said, which can have serious consequences.
“A lot of research has shown that large language models and AI applications are not just producing neutral information, but they also actually can produce very biased information, depending on how they were trained and implemented,” said Williams-Ceci, who this summer will co-teach a course for high school students at Cornell Tech called Interacting with AI: Understanding the Relationship Between AI and Humans. “By doing that, there’s a risk that these systems, inadvertently or purposefully, induce people to write biased viewpoints, which decades of psychology research has shown can in turn shift people’s attitudes.”
Other co-authors are Advait Bhat, a doctoral student at the Paul G. Allen School of Computer Science and Engineering at the University of Washington; Cornell doctoral student Kowe Kadoma; and Lior Zalmanson, associate professor in the Coller School of Management at Tel Aviv University.
Support for this work came from the National Science Foundation and the German National Academic Foundation.