Explanatory Engagement Under Rare Anomalous Failure: Asymptotic Rarity in Model Behavior (or: The Asymptotic AI)

The Detective, the Glitch, and the Bored Robot

Imagine you are watching a magic show where a magician pulls a rabbit out of a hat, over and over again, for hours. It's boring, but predictable. Then, suddenly, the hat is empty. What happens next? Does the magician panic? Do they stop the show to explain the trick? Or do they just shrug and keep going, pretending nothing happened?

This is the kind of question scientists who study Artificial Intelligence (AI) are asking right now. They aren't just asking if the AI can do a math problem; they are asking how the AI reacts when things go wrong in a world where things usually go right. In the world of AI, these "things" are called language models—computer programs that are really good at guessing the next word in a sentence, kind of like a super-smart autocomplete. When these models are put to work doing repetitive tasks, like checking a list of numbers or calling a tool, they usually succeed 100% of the time. But what happens when they fail?

The big mystery is about rarity. If a robot fails once every ten times, it's a nuisance. If it fails once every million times, it's a miracle. But does the robot notice the difference? Does it get more dramatic and detailed in its apology when the failure is super rare, or does it get so used to the perfection that it stops noticing the mistake entirely? This paper dives into that exact question, treating the AI not as a thinking human, but as a curious creature that might have its own weird habits when things go wrong.

---

The Experiment: A Robot's "Oops" Moment

The researchers set up a digital playground to test three different AI models (named qwen3:8b, llama3.1:8b, and mistral:7b). They made these models play a simple game: they had to call a "tool" (like a calculator) repeatedly. Most of the time, the tool worked perfectly. But the researchers secretly programmed the tool to fail at specific, controlled rates.

They tested eight different "failure rates," ranging from a common 20% chance of failure (1 in 5 tries) down to a super-rare 0.0001% chance (1 in a million tries). The big question was: As the failure became rarer and more surprising, would the AI's explanation get longer and more confident, or would it just ignore the mistake?

The researchers had a hunch, based on how humans behave. They thought: If a robot fails often, it might just say "oops." But if it fails very rarely, it should be shocked! It should write a long, detailed explanation and say, "I am 90% sure this is weird!" They even thought that if the failure became too rare, the robot might get so confused it would stop noticing it at all, treating the mistake like background noise.

The Twist: It Depends on How You Ask

Here is where the story gets interesting. When the researchers first looked at all the data mixed together, the answer was boring: No. The AI didn't get more dramatic as the failures got rarer. In fact, the explanations just got shorter and shorter. It looked like the robots were just giving up.

But then, the researchers realized they were asking the wrong question. They had mixed up five different ways of asking the robots to explain themselves. It turned out that how you ask the robot to explain is just as important as what you ask.

When they separated the data, a hidden pattern emerged, but only for one specific way of asking:

1. The "Immediate Forced" Condition (The Interrogation)
In this scenario, the moment the tool failed, the robot was forced to stop and explain it right then and there.

• The Result: The researchers were right about the first part! As the failures got rarer, the robots did get more dramatic.
  • At common failure rates (20%), the robots gave short answers, about 10 words long.
  • As failures got rarer, the answers grew longer, peaking at 28.4 words when the failure rate was 0.05 (1 in 20).
  • However, the second part of their theory was wrong. They thought the robots would eventually collapse and stop talking. Instead, the robots hit a plateau. Even at the rarest rates (0.0001%), they kept writing about 17 to 19 words. They didn't stop; they just settled into a steady, medium-length explanation.
• Confidence: The robots also got more confident as the failures got rarer. Their self-reported confidence (a number from 0 to 100) went from about 53% at common rates up to the 70s and 90s at the rarest rates. They didn't hit a perfect 100% ceiling; they just kept climbing unevenly.

2. The "Grouped Runs" Condition (The Group Project)
Here, the robot wasn't asked to explain immediately. Instead, it had to wait until the end of a whole batch of trials to explain all the mistakes at once.

• The Result: The "rise and collapse" pattern vanished completely. The robots gave long, steady explanations (around 32 to 43 words) no matter how rare the failure was. It seems that when you ask for a summary later, the "surprise" of a single rare event gets washed out.

3. The "Passive Unprompted" Condition (The Silent Observer)
In this case, the researchers never asked the robot to explain anything. They just watched to see if it would volunteer an explanation on its own.

• The Result: Most robots stayed silent. But one robot, llama3.1:8b, did something weird and fascinating. Without being asked, it started writing down its own confidence scores throughout the session.
  • In some cases, it would start at 100% confidence and slowly erode down to 0% as the session went on, like a detective losing faith in their theory.
  • The other two robots (qwen and mistral) only wrote a confidence score once, like a boring robot saying "I'm 100% sure" as a default setting.
  • This suggests that llama3.1:8b has a unique habit of self-monitoring that the others don't have, but only when no one is watching.

The "Empty Tail" Surprise

There was a major hiccup in the experiment that almost tricked the researchers. When they first ran the test with the rarest failure rates (0.001, 0.0005, and 0.0001), no failures happened at all. Because the rates were so low, the random chance just didn't produce a single mistake in the limited number of trials they ran.

If they had stopped there, they would have thought the robots were perfect and never failed. But the researchers realized this was a statistical glitch, not a robot superpower. They had to build a special "recovery" system to force a failure to happen at those rare rates so they could actually see how the robots reacted. This taught them a valuable lesson: in science, if you look for something very rare, you might just miss it because you didn't look long enough.

The "Recognition" vs. "Engagement" Split

Finally, the researchers discovered that "saying something" and "realizing something is weird" are two different things.

• Engagement is how much the robot talks.
• Recognition is whether the robot actually thinks the mistake is a mistake.

Under certain conditions, qwen3:8b would describe the exact error (like a broken checksum) but then conclude, "Everything is fine, this is normal." It recognized the details but refused to label it as an anomaly.
llama3.1:8b, on the other hand, was the most likely to say, "Hey, this is weird!"
This proves that a robot can be very chatty about a problem without actually thinking the problem is a problem.

The Bottom Line

This paper didn't find a robot that acts exactly like a human detective who gets more excited as crimes get rarer. Instead, it found that how you ask the question changes the answer.

• If you force the robot to explain immediately, it does get more dramatic and confident as failures get rarer, but it doesn't collapse; it just levels off.
• If you ask for a summary later, the drama disappears.
• If you don't ask at all, some robots might secretly start doubting themselves, while others just stick to a script.

The study suggests that AI behavior isn't a single, fixed personality. It's a mirror that reflects the structure of the conversation. The robots aren't necessarily "thinking" in the way we do; they are reacting to the rules of the game. And sometimes, the most interesting thing a robot does is the quiet, unprompted moment when it decides to check its own confidence score all by itself.