AI persuasion, considered

There was some research news about this last year, but Science has got a lengthy article up about the topic now, and it's a good read:

Powers of persuasion

AI chatbots are becoming experts at changing people’s minds. What gives them an edge?

The real test for them would be if they can talk sense into MAGA Trump cultists.  But I don't expect them to be that powerful.  (Or, maybe I'm wrong - see below!)

Anyway, due to the perpetual problem of link rot, here are some good extracts:

...She was part of a study in which more than 2000 people debated either a chatbot or a human about political issues, ranging from a social media ban for teenagers to assisted suicide. When Kobi Hackenburg, an AI researcher at the University of Oxford who led the study, posted a preprint on the results in June, they were sobering: No matter whether it was ChatGPT, Google’s Gemini, or Claude, the AI was consistently better than humans at swaying the other participants. “To my mind, this is already a landmark publication in the fields of political persuasion and AI and human behavior,” says Robb Willer, a sociologist at Stanford University who was not involved in the work. 

Hackenburg’s paper is the latest in a string of studies showing the power of AI to sway people. “It’s a whole new field that is emerging,” says Sander van der Linden, a psychologist at the University of Cambridge. “People are very interested in the persuasive powers of AI, I think, both for ethical and unethical reasons.” As the field gathers steam, it is raising a host of theoretical and practical questions. How exactly do chatbots win over people? (Warning: Lying is one answer.) How much better could they get? And who will control them?

As well as other studies, this one is mentioned:

...in September 2024, Tom Costello, a psychologist at Carnegie Mellon University, and colleagues published a Science paper showing that ChatGPT could even persuade people out of conspiracy beliefs. In the experiments, participants described a conspiracy theory that they believed in, from the U.S. government being behind the 9/11 attacks to the British royal family orchestrating Princess Diana’s death, and then had a three-round conversation on it with the chatbot. On average, participants’ embrace of their chosen conspiracy theory declined by almost 17 points on a 100-point scale. 

Other researchers were stunned. Conspiracy beliefs are notoriously difficult to change. “Nothing had ever worked in that space,” van der Linden says. (After mistakes in the public data set and analysis pipeline were found, the paper will have a correction, but the authors say the new results match those of the original paper in size and direction.) Even the researchers themselves were taken aback. “I was skeptical when we first started in terms of how effective it would be,” says Gordon Pennycook, a psychologist at Cornell University and author on the paper. “But it blew us out of the water. We were shocked when we saw the results.”

Interestingly, though, it seems chatbots who are trained to be more persuasive do so by becoming more like politicians (economical with the truth): 

Like Willer in his foundational study, other researchers have shown that AI’s powers rely on appeals to facts and evidence. In a 2025 preprint, a follow-up to their Science paper, Costello and his colleagues found that the only time the chatbot was unsuccessful in convincing people out of conspiracy theories was when it was forbidden from using evidence or rational arguments. “It tries to say, ‘Oh, well, you shouldn’t believe this. This is really damaging, and it could hurt people,’” Pennycook says. “People are like, ‘You haven’t given me any reasons to change my mind.’ And so they don’t change their mind.”

The study confirms people will listen to good arguments, Pennycook says. “Facts and evidence really matter.” But this doesn’t mean the facts used by chatbots necessarily have to be accurate. In a Science paper published last year, Hackenburg found that models trained to become more persuasive also ended up being less truthful. It’s possible the models learn that “facts” seem to be the thing that most persuades people, Hackenburg says, and end up filling their conversations with dubious or simply false ones. “They start scraping the bottom of the barrel of the facts that they have and know and so the quality of facts just sort of degrades,” he says. Even in Hackenburg’s recent preprint, Claude spouted numerous inaccuracies and falsehoods when persuading the U.K.-based participant to support tougher penalties for disruptive protests.

And, they also may be persuasive because they are automated Gish Gallopers!:

 But the biggest advantage AI has over humans seems to be sheer speed. In his Science paper, Hackenburg and his colleagues found that the number of fact-checkable claims in a conversation predicted how persuasive it was and they suggested AI’s edge might come from writing much more text containing more claims in a shorter period of time. Indeed, when Hackenburg ran his competition of coached elite debaters and AI, there was one way he could bring AI down to human levels of persuasiveness: by forcing it to write human-length messages at human writing speed.

I guess this all points to robot politicians being very successful in future!