Why do we need social singularity? A mechanism-based critique of gradual scenarios in AI existential-risk discourse

The Great AI Rollercoaster: Why the Future Isn't a Straight Line

Imagine you are watching a movie about the future of Artificial Intelligence (AI). For a long time, the most popular plotlines have been two very different kinds of disasters. The first is the "sudden crash": a super-smart robot wakes up one morning, decides humans are obsolete, and takes over the world in a flash. The second is the "slow fade": we build better and better tools, and slowly, without anyone noticing, we hand over all our power to them until we are just passengers in our own lives. Both of these stories assume that once we start building these machines, they will just keep going, and society will just keep watching, unable to stop the train.

But there is a third possibility that often gets ignored: the idea that the story of the future might change the future itself. This paper, written by Petr O. Jedlička, dives into the messy, chaotic, and unpredictable world of human society to ask a simple question: What happens when people get scared, angry, or excited about AI before it even becomes super-smart? The author argues that we can't just look at the technology like a math problem. We have to look at the people. Just like a rollercoaster doesn't just follow the track; it depends on whether the riders scream, hold on tight, or try to jump off. If the people on the ride panic or fight back, the ride might stop, change direction, or crash long before it reaches the top. This paper suggests that the biggest danger (or the biggest hope) might not come from the AI itself, but from how we react to the idea of it.

The Paper's Big Idea: The "Social Singularity"

The paper starts by critiquing a popular group of stories about AI risk called "gradual disempowerment." These stories, written by thinkers like Kasirzadeh and Kulveit, imagine a future where AI slowly takes over our jobs, our governments, and our decisions. They picture a "boiling frog" scenario: the water gets hotter and hotter, but the frog (us) never jumps out until it's too late. The author argues that these stories are missing a huge piece of the puzzle. They treat society like a passive movie screen where the AI projector just shines its light, ignoring the fact that the audience can throw tomatoes, turn off the projector, or even storm the stage.

The author calls this missing piece "technological determinism." It's the belief that technology has its own mind and will march forward no matter what. The paper suggests this is wrong. History shows us that when new inventions arrive, people don't just sit there. They argue, they organize, they break machines (remember the Luddites?), and they make new rules. The paper proposes that instead of a smooth slide into a robot-run world, we are more likely to see a messy, bumpy ride full of conflicts, arguments, and sudden stops.

The Engine Room: How Society Actually Works

To explain why the "slow fade" story is unlikely, the author introduces a toolbox of "social mechanisms." Think of these as the hidden gears that make society move, often in ways that surprise us.

1. The Power of Storytelling (Interpretation and Performativity)
Imagine a rumor starts that a bank is going broke. Even if the bank has plenty of money, if everyone believes it's broke and runs to withdraw their cash, the bank will actually go broke. The belief created the reality. The paper says this happens with AI too. If experts and the public start telling stories that "AI is a monster," people will act like it is, creating laws and protests that might stop it. If they tell stories that "AI is a savior," they might rush to build it, ignoring the dangers. The way we talk about AI changes what AI actually does.

2. The Great Fight (Mobilization and Conflict)
The paper points out that when people feel threatened, they don't just give up; they fight back. Just like workers in the past broke machines to save their jobs, we might see "AI-resistance movements" today. On one side, you might have "AI-loyalists"—companies and workers who love the new tech and want more of it. On the other side, you could have "anti-AI radicals" who feel left behind or scared. These two groups might clash, creating a political storm that could stop the smooth rollout of AI. The paper suggests that the future isn't a straight line; it's a battlefield where different groups are fighting to decide which way the technology goes.

3. The Different Roads (Path Dependence and Divergence)
Imagine a group of friends trying to walk to a destination. If they all agree on the map, they walk together. But if they are in different countries with different rules, they might end up on totally different paths. The paper notes that the US, China, and Europe have very different laws, cultures, and economies. The US might race ahead with wild, unregulated AI, while Europe might build a slow, safe, but boring version, and China might build a version that follows strict government rules. Because of these differences, AI won't spread evenly around the world. It will be a "multi-speed" race, creating a messy world where some places are full of advanced AI and others are just starting.

4. The Heart and Soul (Social-Psychological Mechanisms)
This is where it gets personal. The paper looks at how we might start to love or hate our AI friends. Some people might form deep emotional bonds with AI companions, treating them like family (a bit like a child bonding with a robot toy). This could make them want to protect AI. But others might feel their freedom is being stolen. If an AI tries to tell them what to do, they might get angry and rebel (a feeling called "reactance"). This emotional tug-of-war could split society into two camps: those who want to hug the robots and those who want to smash them.

The Big Twist: The "Social Singularity"

Here is the paper's most exciting idea. Usually, people talk about the "Technological Singularity"—the moment when AI becomes so smart it changes everything. But the author suggests we might hit a different kind of singularity first: the Social Singularity.

Think of it like this: The "Technological Singularity" is the moment the rocket actually reaches space. The "Social Singularity" is the moment everyone on the ground starts screaming, "Wait! We aren't ready!" and tries to cut the fuel lines. The paper argues that the anticipation of the AI explosion is so powerful that it will cause massive social changes before the AI even gets super-smart.

We might see a "Social Singularity" where society reacts so strongly to the idea of AI that it changes the technology's path. Maybe we ban it, maybe we slow it down, or maybe we fight a war over it. The paper suggests that the most dramatic event in our future might not be the AI taking over, but the humans fighting over whether to let it take over.

What This Means for the Future

The paper doesn't say that AI won't be dangerous. It doesn't say we will definitely stop it. Instead, it says that the future is much more complicated than the "slow fade" stories suggest. It's not a straight line where we just lose control. It's a branching tree with many possible paths.

• It's not a done deal: The future isn't written in stone. The paper suggests that because of human conflict, different laws in different countries, and our emotional reactions, the path of AI will be full of twists, turns, and sudden stops.
• It's messy: We might see a world where some places have advanced AI and others don't, or where AI is banned in some cities but free in others.
• It's up to us: The most important factor isn't just the code the programmers write; it's the arguments, the protests, the laws, and the feelings of the people living with the technology.

In short, the paper tells us to stop imagining the future as a movie where the robots win or lose on their own. Instead, we should imagine it as a giant, chaotic game of tug-of-war, where the rope is the future of AI, and we are all pulling on it, trying to decide which way it goes. The "Social Singularity" is the moment the game gets so intense that the whole world changes, not because the robots became smart, but because we finally decided to fight back.