AI's Fiercest Critics Are Opting Out of the Conversations That Matters Most

This week I’m publishing a guest post by my PhD student, Victória Oldemburgo de Mello.

Victória is the first author of a new working paper called The Moralization of Artificial Intelligence. I adore this paper. So much so that I want to claim it as all my own. I talk about it at conferences and invited talks all the time because it illuminates something real about the moment we’re living in. Conversations about AI have stopped being normal conversations. They’re fever-pitched, with people holding strong opinions they sometimes refuse to interrogate. Otherwise smart people lose their minds if someone dares say something appreciative about a technology that is, despite all the hype and doom, normal technology.

Victória’s paper explains why. She shows how attitudes toward AI have shifted from mere preferences into something moral, something good-versus-evil. I’ll say no more, because her own words are all you need.

Before I go, a few words about Victória herself. She is not merely brilliant. She is broadly interested in the world—from opera to technology from ethics to cognitive science. More important, she has a nose for good ideas, something that cannot be taught. She also lives well, valuing hard work and pleasure in equal measure. I credit her with getting me genuinely interested in AI. I also credit her with something I underappreciated before she joined our lab: a thriving lab needs a social connector. Victória was the lab’s glue. She made it obvious how much it matters to have someone who connects people, who makes a group feel like a group. Victória is starting a postdoc in a few months. And I’m missing her already.

---

“We cannot be satisfied with merely calling for the moralization of machines—the so-called ‘alignment’ of AI with human values—without also having the courage to insist on a further condition: the possibility of openly discussing the ethical frameworks involved and subjecting them to shared standards of social justice. […] What is needed is a more active political involvement that is capable of slowing things down when everything is accelerating, and of protecting the opportunities for communities still to be able to participate and ask questions.”

So wrote Pope Leo XIV in Magnifica Humanitas, his recent encyclical on artificial intelligence.

If you’re chronically online like me, you probably saw the excerpts make the rounds, recast as proof that even the Pope had turned against AI. For a certain crowd, his words confirmed what they already believed: that this technology is a kind of demon to be cast out, and the only faithful response is to bring it to a halt.

But read the whole memo and you’ll notice that the pope’s stance is not an antagonistic one, but a cautious one. He acknowledges that AI is being built and embedded into our lives, as many other technologies before it. The task is not to escape it, but to take a seat at the table where its development and deployment is discussed.

Here’s the irony: the people who most need to be part of this discussion are the least likely to pull up that chair and sit down at that table. Not because they don’t care about AI, but because they refuse to share a table with the people building it: they’ve moralized it.

In a recent paper, my co-authors and I examine whether some of the AI opposition we’re seeing is moral in nature. In this post, I’ll walk through our findings and show that for many AI skeptics, moralization is a central driver of their opposition.

But before the findings, let me say more about what moralization actually is, and why it can be such a problem. (If you’ve read your lifetime’s fill of papers on moral conviction, skip ahead to the Findings.)

What Makes a Moral Belief Different

Let’s start with a distinction that seems obvious until you poke it.

You like pineapple on pizza. I think pineapple on pizza is an abomination. We argue about it for a while, and eventually one of us says “look, it’s just a matter of taste,” and we both relax, because it is just a matter of taste, and we both know it.

Now replace the pizza with the concept of animal torture.

You think torturing animals for fun is fine. I think it isn’t. Here, I do not reach for “well, it’s just a matter of taste.” I don’t think we have different preferences the way we have different opinions about pizza. I think you’re wrong and evil. There’s a fact here, the fact has a correct answer: it’s simply wrong to torture animals, and you’ve got it backwards.

Now imagine you mention to a colleague, the way you’d mention the weather, that you got your kids their flu shots over the weekend. To you this has the moral weight of a dentist appointment: a chore, a copay, a lollipop on the way out. Your colleague hears something else. To them you’ve just confessed to poisoning your children on the orders of people who profit from it, and they are now recalculating everything they thought they knew about you.

A strange feature of moral beliefs is that they don’t feel like preferences at all: they feel like facts. When two people disagree about a moral question, each one tends to quietly assume the other must be mistaken—not just different, mistaken. Thus, the way we think about morality is closer to arithmetic (where there’s one correct answer and many incorrect ones) than something like pizza toppings. Now add the fact that people don’t even agree on which questions are matters of arithmetic and which are matters of pizza, and you get two people in the same conversation, one making small talk, the other witnessing a crime, neither aware they’re playing different games.

Morality is also strongly tied to your identity. Here’s a thought experiment: Imagine you woke up tomorrow having forgotten last summer—just gone, a blank. Are you still you? Sure, mostly. Now imagine you woke up tomorrow believing that cruelty is good and kindness is contemptible. Are you still you? When people get asked what’s most essential to who they are, moral character beats memory and personality. Your values aren’t a feature of you; they’re you. Which is why changing a moral belief is so hard; it registers as becoming a different person.

Moral values don’t bend to the crowd. Humans are deeply, embarrassingly conformist. But here’s an exception: morality! You can tell someone that everyone disagrees with them, that the experts disagree, that the majority is overwhelming—and on a moral question, people simply refuse to bend[1].

Moral values can’t be priced. As many economists will tell you, humans operate on a constant utility calculus: Let me figure out how much each option is worth, and I will pick the one that yields the best value for me. But try to put a number on a moral value and the machinery falls apart. Ask someone what dollar amount would make it okay to sell out their best friend, and you will not get a number; you’ll get a frown. The question itself reads as offensive—as if the willingness to compute the answer is already a kind of betrayal. The mere suggestion of a tradeoff produces not a utility calculation but disgust.

So put it together. A moral belief feels true the way math feels true. It’s wired into your sense of who you are. It ignores the crowd. And it refuses to be put on a scale and weighed against anything else.

You might reasonably ask: so what? People hold some beliefs more tightly than others. Why should anyone care which beliefs sit in the “moral” bucket?

Because the bucket predicts important real-world outcomes.

Why the moral bucket matters. Moral conviction corrodes our ability to tolerate one another. Disagree with someone about pizza toppings and they think you’re weird. Disagree on a moralized topic and they want you gone—they want physical and social distance from you, from people like you, and from information that sounds like you.

In practice, that means they avoid people like you and act in ways that shut you out: they’re more likely to vote, donate, march, and protest alongside people who think as they do—and not as you do—even when they have nothing personally at stake.

Follow that logic to its end and you arrive somewhere dark. Moral conviction predicts not only a desire for distance but a greater acceptance of violence as a way to resolve conflict. I want to leave that thought hanging over everything that follows—because resistance to AI has already begun producing more than op-eds and think-pieces. It has produced threats, break-ins, and worse.

Quietly, moralization sorts a society into camps that would rather not share a room—or a planet. And we already have another name for that: polarization.

Our Findings

So how do we know whether an issue is moralized?

You measure it. And the most direct way is to ask. Moralized attitudes have recognizable fingerprints, so you get them by asking people a series of questions. Do you think you’re objectively right? Is the view tied to who you are? If most people came to disagree with you, would you still hold it? Is the thing wrong only in your community, or everywhere? And if the harms were mostly fixed, would you change your mind—or not?

In three studies, my co-authors and I surveyed a representative sample of Americans about a range of AI applications: chatbots, AI companions, legal AI, AI doctors, autonomous vehicles, and more. For each one, we asked whether they opposed that specific use of AI and whether they endorsed those hallmarks of moral conviction.

The first surprise: most people aren’t AI opponents at all. Opposition ranged from 13% to 45% depending on the application—a minority in every case. But among the people who did oppose, a stark majority endorsed every hallmark of moralization. So if you pull a random American, you’ll probably get someone who’s fine with AI. But if you pull from the opponents, you’ll very likely to get someone who strongly moralizes it.

Then we wanted to know whether moral opposition actually changes what people do—not just what they say. Specifically, whether it would make people refuse AI even when using it would benefit them personally. (Remember the bit about moralized attitudes refusing the cost-benefit tradeoff?)

So we ran an experiment. We brought back participants from the earlier surveys and gave them a student essay to grade—assign a score from 1 to 10, write some feedback. We told them they could use GradeAI, a tool we described as more accurate than human graders, to help. And here’s the twist: we told them the essay had already been graded by an experienced teacher, and if their score landed within 5 percentage points of the teacher’s, they’d earn a cash bonus. In other words, we offered to pay them to use the AI to their own advantage.

Moral opposition predicted refusal. A one–standard-deviation increase in AI moralization predicted a 42% drop in the likelihood of using the tool. Put differently: the heaviest moralizers (top 25%) were nearly three times more likely to avoid using the AI than the lightest (bottom 25%[2]. In plain words: AI moralizers chose to forgo extra money to act on their convictions.

Moralization in culture

We’ve shown that individuals moralize AI. We also wanted to know whether you could see the same thing at a collective level—in the culture rather than in the survey responses of single people. So we turned to the news, and to how journalists write about AI.

This was possible thanks to a researcher who built a large corpus of news headlines and made it freely available, and to others who developed methods for detecting moral language in text. We borrowed both.

We measured how much moral language surrounds a range of topics in news coverage. Some we already knew are moralized, like abortion and climate change. Some became moralized over time, like vaccines, GMOs, and COVID. And one we expected to be essentially neutral—a control—interior design.

Here are some example AI headlines. A highly moralized one: “Living Nostradamus warns AI is ‘digital antichrist ready to corrupt our innocent souls’”. Here’s one that scored very low in moralization: “Meet GERI, the artificial intelligence personal coach.”

AI ranked as the third most moralized topic in the headlines, behind only abortion and climate change. It drew moral language at roughly the same rate as GMOs and COVID-19, and—notably—at a statistically higher rate than vaccines, a topic with a deep, well-documented history of moralization.

Moralization of technology and social topics in news headlines from 2018 to 2024. Mean moralization for each topic. Error bars represent standard errors.

The key takeaway

So: most AI opponents moralize AI, and that moralization predicts behavior, and that’s visible not just in individuals but in the culture itself, written into the way we talk about the technology in the news. These findings have real consequences for how the AI debate is going to unfold.

The biggest one is this: when a disagreement becomes moral, it stops being the kind of disagreement you can resolve. Moralized opinions tend to make honest debate difficult, if not impossible. The usual debate + compromise mechanism runs on each side’s willingness to trade with each other. And moral values, as we’ve seen, don’t tolerate tradeoffs. Once a position becomes a moral one, the ordinary principles of good-faith dialogue start to fall apart, replaced by psychological earmuffs: a refusal to listen to, or engage with, anyone on the other side.

And that’s what I find genuinely worrying about moralization—not that it makes people care deeply, but that it may make good AI policy harder rather than easier. The people best positioned to research the risks, to develop guardrails, to ask the uncomfortable questions, are the least likely to truly engage in these discussions. In practice, that might look like ad hominem attacks—discrediting the technology based on its creators or proponents. It might also look like misguided claims about its harms. Not because concerns are unfounded, but because moralizers may latch onto confirming evidence so tightly that they lose sight of its limitations, leaving the real problems without serious advocates[3].

Take the water argument, for instance. AI critics often cite data center water consumption as evidence of environmental catastrophe—and stop there. The claim may be more contested than it appears, but let’s grant it a real risk: data centers can lead to localized water bottlenecks that may cause price surges and shortages. That is a real concern. But the conversation rarely gets to the solutions already being discussed—closed-loop cooling systems that recycle rather than discharge water, alternative coolants that don’t use water at all, taxes structured so any increase in costs don’t pass through to nearby residents, or requirements that data centers build their own infrastructure (an approach already being piloted for energy consumption through small modular reactors in the U.S. and Japan).

To be clear: AI problems are real, and they are far from solved. But dismissing a technology without reckoning with these conversations isn’t protecting the future; it’s opting out of it.

Which brings us back to where we started, and to the Pope’s quietly radical ask: that we don’t settle for the moralization of machines, and instead protect people’s ability to participate and ask questions. The hardest part of that isn’t getting AI’s champions to build responsibly. It’s getting its fiercest critics to stay in the room long enough to both listen and be heard[4].

---

[1] Some of you may be asking, “isn’t being an independent thinker a good thing?” I’d agree it can be. But conformity, too, can be healthy and even necessary for a society to function.

[2] If you’re interested, we have some robustness analysis in the paper where we controlled for a bunch of things, including how easy participants found the task and familiarity with AI. Results remained robust to all of that.

[3] Worth flagging that some proponents probably moralize AI too, as moral supporters. I don’t have the data yet, but I’d bet a pinky that at least some of the drive to build is powered by people who feel a moral imperative to do it. Moralization isn’t only the opponents’ problem; it just bites hardest where it shuts down dialogue.

[4] Thanks to Michael Inzlicht and Gabriel Almeida Prado for feedback on drafts of this.