💬 NLP

Even GPT-5.2 Can't Count to Five: The Case for Zero-Error Horizons in Trustworthy LLMs

This paper introduces the Zero-Error Horizon (ZEH) metric to evaluate the maximum error-free reasoning range of large language models, revealing that even state-of-the-art systems like GPT-5.2 fail on fundamental tasks while demonstrating ZEH's utility in analyzing algorithmic emergence and proposing efficient computational methods to scale its evaluation.

Original authors: Ryoma Sato

Published 2026-01-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ryoma Sato

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a brilliant, super-intelligent assistant to help you run a nuclear power plant, manage a bank, or perform surgery. This assistant can write complex computer code, explain quantum physics, and hold a conversation about art. But there's a catch: this assistant sometimes fails at things a 5-year-old could do.

The paper "Even GPT-5.2 Can't Count to Five" introduces a new way to test these AI assistants called Zero-Error Horizon (ZEH). Think of ZEH not as a test score, but as a "safety fence."

Here is the breakdown of the paper's findings using simple analogies:

1. The "Safety Fence" Concept (What is ZEH?)

Most people test AI by giving it a mix of easy and hard questions and counting how many it gets right (like a grade of 95%). The authors say this is dangerous for safety-critical jobs. If an AI gets 99% of math problems right, but fails on one specific number, that one failure could cause a disaster.

ZEH is the "fence" around the AI's perfect zone.

Inside the fence: The AI is guaranteed to be 100% correct.
Outside the fence: The AI might make a mistake.
The Fence Height: If an AI has a ZEH of 10 for multiplication, it means it can perfectly multiply any numbers up to 10. But the moment you ask it to multiply 11, it might fail.

2. The Shocking Discovery (The "Foolish" Mistakes)

The authors tested a very advanced model (GPT-5.2) and found its "fence" was surprisingly low for simple tasks.

The Parity Test: Can the AI count if a string of numbers like 11000 has an odd or even number of 1s? The AI failed on this 5-digit string. Its fence was at 4.
The Parenthesis Test: Can the AI tell if ((((()))))) has balanced brackets? The AI said "Yes" when it was actually "No."
The Math Test: The AI could write complex physics simulations but got the simple math problem 127 × 82 wrong.

The Analogy: Imagine a master chef who can create a 10-course gourmet meal but accidentally drops a single grain of salt on the floor while making a sandwich. In a normal kitchen, you'd ignore the grain. In a nuclear reactor or a bank, that single grain of salt (or wrong digit) could be catastrophic.

3. Why ZEH is Better Than a "Report Card"

The paper argues that standard accuracy scores are like cherry-picking a menu.

The Trap: A researcher could say, "Our new AI is great! It got 99% of multiplication problems right!" But they might only be testing numbers up to 20. If you test up to 100, the AI might fail miserably.
The ZEH Solution: ZEH doesn't let you pick the menu. It forces the AI to prove it can handle everything up to a certain point. If the AI fails on number 13, the fence is set at 12. No matter how high the "average score" is, the fence tells you exactly where the danger zone begins.

4. How AI "Learns" (Memorization vs. Understanding)

The authors studied a family of models (Qwen2.5) of different sizes to see how they improve.

Small Models (The Photocopiers): Small AI models rely on memorization. They have "seen" the answer 2 × 2 = 4 in their training data. But if they haven't seen 2 × 3, they might guess wrong. Their "fence" is shaky and full of holes.
Large Models (The Mathematicians): As the models get bigger, they stop just memorizing and start learning the rules (algorithms).
- The Clue: The authors found that larger models make "structured" mistakes. For example, if the answer is 986, a large model might say 1006. The difference is exactly 20. This looks like a human making a "carry-over" error in long multiplication.
- The Result: This proves the large model is actually doing the math, not just guessing. Because it's using rules, its "safety fence" (ZEH) grows steadily and predictably.

5. The Cost of Checking the Fence

The authors admit that checking this "fence" is expensive. To find the exact point where the AI fails, you have to test every single problem starting from 1, 2, 3... all the way up until it fails.

The Solution: They developed a "speed-up" toolkit (using tree structures and smart caching) that makes this testing process up to 10 times faster. It's like having a robot that can check every step of a staircase in seconds instead of walking them one by one.

Summary

The paper argues that for safety-critical jobs (like finance or medicine), we shouldn't just ask, "How smart is this AI?" We should ask, "Where is the exact line where this AI stops being perfect?"

The Zero-Error Horizon is that line. It reveals that even the smartest AIs have "blind spots" on surprisingly simple tasks, and it gives us a concrete, mathematical way to know exactly how far we can trust them before they start making mistakes.

Technical Summary: Zero-Error Horizons in Trustworthy LLMs

1. Problem Statement

Large Language Models (LLMs) are increasingly deployed in safety-critical domains such as healthcare, finance, and nuclear systems. Despite their apparent reasoning capabilities, LLMs exhibit "holes" in their competence, making inexplicable errors on simple tasks (e.g., calculating $127 \times 82$ , determining the parity of a short binary string, or checking balanced parentheses).

Current evaluation metrics, primarily accuracy, are insufficient for safety-critical applications for several reasons:

Scale Arbitrariness: Accuracy depends on the evaluator's choice of problem range (e.g., testing multiplication up to $50 \times 50$ vs. $99 \times 99$ ). A model may appear highly accurate on a limited range while failing catastrophically on slightly larger inputs.
Lack of Safety Guarantees: High average accuracy does not preclude the existence of "holes" where the model fails. In safety-critical contexts, a single error can be fatal.
Saturation: Fixed-size benchmarks eventually saturate as models improve, losing their ability to differentiate state-of-the-art performance.

The paper argues that there is a need for a metric that defines a guaranteed safe boundary rather than an average performance score.

2. Methodology: Zero-Error Horizon (ZEH)

The authors propose Zero-Error Horizon (ZEH), a metric representing the maximum problem size $n$ for which a model can solve all instances without error under deterministic greedy decoding.

Definition

Given a fixed model, problem type, and prompt:

Let $B_n$ be the set of problems of size $n$ .
Let $T_n = \bigcup_{i=1}^n B_i$ be the set of all problems of size at most $n$ .
Let $C$ be the set of problems the model answers correctly.
ZEH is defined as:
$\text{ZEH} = \max \{ n \mid T_n \subseteq C \}$

In essence, a model with $\text{ZEH} = k$ solves every instance of size $1$ through $k$ correctly but makes at least one error on an instance of size $k+1$ . The specific instance of size $k+1$ where the model first fails is termed a ZEH Limiter.

Evaluation Tasks

The authors evaluate ZEH on four tasks:

Multiplication: Integer multiplication ( $a \times b$ ), where size is defined as $\max(a, b)$ .
Parity: Determining the parity (even/odd count of 1s) of binary strings, where size is string length.
Balanced Parentheses: Determining if a string of parentheses is balanced, where size is string length.
Graph Coloring: Finding the chromatic number of simple undirected graphs, where size is the number of nodes.

Optimization for Efficiency

Evaluating ZEH requires exhaustive verification, which is computationally expensive. The paper proposes four acceleration techniques:

Teacher Forcing: Instead of autoregressive decoding, the model is fed the full sequence (e.g., $28 \times 43 = 1204$ ) with a causal mask to verify token-by-token correctness in parallel.
Look-Ahead (Batching): Instances of different sizes are batched together for parallel processing to maximize GPU utilization, rather than processing size-by-size.
Prompt Prefilling: Common prompt prefixes are processed once, and their Key-Value (KV) caches are shared across all instances.
FlashTree: A tree-structured approach using a Trie to share KV caches for shared prefixes among instances (e.g., $1 \times 1$ , $1 \times 2$ , $1 \times 3$ ). This utilizes an online softmax implementation (inspired by FlashAttention) to avoid the $O(N^2)$ attention mask bottleneck, achieving up to a 10x speedup over naive autoregression.

3. Key Results

3.1 Surprising Failures in State-of-the-Art Models

The evaluation of GPT-5.2 (a hypothetical model in the paper's context, dated Jan 2026) reveals significant capability gaps:

Parity: ZEH = 4. The model fails on the 5-character string 11000.
Balanced Parentheses: ZEH = 10. The model fails on the 11-character string ((((()))))).
Multiplication: ZEH = 126. The model fails on $127 \times 82$ .
These results demonstrate that even models capable of complex code generation (e.g., fluid dynamics) fail on elementary arithmetic and logic tasks.

3.2 Analysis of Qwen2.5 Family

The authors analyzed the Qwen2.5 series (0.5B to 72B) on multiplication:

Correlation with Accuracy: While ZEH and accuracy generally increase with model size, they do not correlate one-to-one. For example, accuracy for the 72B model on $99 \times 99$ is 98.6%, but its ZEH is only 42.
Memorization vs. Algorithmic Reasoning:
- Small Models (0.5B - 3B): Show high correlation between training data frequency (C4 corpus) and accuracy, suggesting reliance on memorization. Their ZEH is low and stagnant because memorization is stochastic; random "holes" in the training data prevent a continuous error-free horizon.
- Large Models (14B - 72B): Show decreasing correlation with training frequency, indicating a shift to algorithmic reasoning.
Structured Errors: As models grow larger, errors become more "structured." For large models, the difference between the predicted and correct answer is often a multiple of 10 (e.g., predicting 1006 instead of 986). This suggests the model understands the multiplication algorithm (e.g., getting the ones digit correct) but makes execution errors (e.g., carry mistakes), rather than random hallucinations.
Carry Sensitivity: Logistic regression analysis shows that larger models are more sensitive to "carry" operations (where digit multiplication $\ge 10$ ) than smaller models. This counter-intuitive result supports the hypothesis that larger models are attempting to perform arithmetic rules rather than retrieving memorized answers; thus, they are susceptible to the computational difficulty of the rules they are trying to execute.

3.3 Prompt Stability

ZEH is relatively robust to prompt variations. For Qwen2.5 models, the standard deviation of ZEH across different prompt phrasings was approximately 3. However, for safety-critical applications, the authors recommend using the lowest ZEH observed across multiple prompts.

4. Significance and Claims

The paper claims that ZEH offers distinct advantages over traditional accuracy metrics for trustworthy LLMs:

Safety Guarantees and Warning Signals: ZEH provides a clear boundary: problems within the horizon are "safe" (guaranteed correct), while problems beyond it are "dangerous." This is crucial for safety-critical domains where average performance is insufficient.
Concrete Evidence (ZEH Limiters): Unlike abstract accuracy scores, ZEH provides specific, reproducible examples (limiters) where the model fails. This facilitates debugging and provides undeniable proof of capability gaps.
Objective Metric: ZEH eliminates the "scale arbitrariness" of accuracy. The horizon is determined by the model's intrinsic capabilities, not by an evaluator's choice of test range.
Insight into Emergence: ZEH is a sensitive indicator of the transition from memorization to algorithmic reasoning. The growth of ZEH correlates with the model's ability to execute rules reliably, even if overall accuracy plateaus.
Future-Proofing: Unlike fixed benchmarks that saturate, ZEH is open-ended and continues to serve as a metric as models improve.

The authors conclude that while ZEH incurs computational costs, the proposed optimization techniques (FlashTree, etc.) make it feasible. They emphasize that ZEH should be used as a complementary metric to accuracy, particularly for verifying reliability in high-stakes applications.