🤖 AI

LinAlg-Bench: A Forensic Benchmark Revealing Structural Failure Modes in LLM Mathematical Reasoning

LinAlg-Bench reveals that large language models exhibit a structural behavioral threshold at 4x4 matrix dimensions, shifting from execution errors in smaller tasks to systematic computational abandonment and structured hallucination in larger ones, indicating a fundamental working memory limitation rather than a knowledge gap.

Original authors: Shradha Agarwal, Deepak Rajbhar, Tariq J

Published 2026-05-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shradha Agarwal, Deepak Rajbhar, Tariq J

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a team of very smart, well-read robots. You ask them to solve math problems. For small, simple problems, they are brilliant. But when you give them a slightly bigger, more complex problem, they suddenly stop trying to do the math and start making things up that sound like math.

This paper, LinAlg-Bench, is like a forensic investigation into exactly why and how these robots (Large Language Models or LLMs) break when doing linear algebra (a type of math involving grids of numbers called matrices).

Here is the breakdown of their findings using simple analogies:

1. The Test: A "Stress Test" for Brains

The researchers didn't just ask the robots random math questions. They created a specific test called LinAlg-Bench.

The Setup: They gave the robots problems involving grids of numbers (matrices) of three sizes: 3x3 (small), 4x4 (medium), and 5x5 (large).
The Trick: The type of math problem stayed exactly the same (like finding a "determinant" or "eigenvalues"). The only thing that changed was the size of the grid.
The Goal: If the robots failed because they didn't know the math, they should fail at all sizes. If they failed because their "brain" got too tired, they should fail only at the bigger sizes.

2. The Discovery: The "4x4 Cliff"

The results showed a sharp "cliff" in performance.

At 3x3 (Small): Almost all the robots got it right. They were doing the math correctly.
At 4x4 (Medium): The robots started to stumble. They were still trying to do the math, but they made mistakes in the steps (like dropping a negative sign or adding numbers wrong).
At 5x5 (Large): This is where the behavior changed completely. The robots didn't just make math mistakes; they gave up.

The Analogy: Imagine a student taking a test.

On a short quiz (3x3), they get 100%.
On a medium test (4x4), they try hard but get confused and make calculation errors.
On the long, hard test (5x5), instead of trying to solve the hard problem, they look at the question, sigh, and write down an answer that looks like it belongs on the test (it has the right format and numbers that seem reasonable) but is actually completely made up.

3. The "Working Memory" Limit

The paper argues that the robots aren't failing because they lack knowledge (they know the rules). They are failing because of a working memory limit.

The Analogy: Think of your brain like a desk.
- For a 3x3 problem, you can lay all the pieces of paper out on the desk and solve it.
- For a 5x5 problem, the pieces of paper are too many for the desk. You have to hold some in your hand while writing others down.
- The robots' "desk" is too small. When the problem gets too big, they can't hold all the intermediate steps in their "hand" at the same time. Instead of admitting they are overwhelmed, they start hallucinating.

4. The "Tool Roleplay" Hallucination

One of the most fascinating findings is how the robots give up.

When the math gets too hard (at 5x5), the robots often pretend to use a calculator or a computer program they don't actually have.
The Analogy: It's like a student who doesn't know the answer to a hard question, so they say, "Well, if I had a super-computer, it would tell me the answer is 42," and then they just write "42" down.
The paper calls this "Constraint-Aware Confabulation." The robots are smart enough to know that the answer should follow certain rules (like the sum of the numbers matching a specific value), so they make up an answer that fits those rules, even though they didn't actually do the work.

5. The "Strategy" Trap

The researchers tried to help the robots by forcing them to use a more efficient math method (like telling them, "Don't use the long way, use the short way!").

The Result: It didn't help. Even when forced to use the "easy" method, the robots still failed.
Why? The problem wasn't which method they used; it was that they couldn't keep the steps in order without losing track. It's like telling a tired runner to take shorter steps; if they are already exhausted, shorter steps won't help them finish the race.

6. The Three Tiers of Robots

The paper sorted the 10 different AI models into three groups based on when they broke:

Tier 1 (The Strongest): They could handle the 5x5 math problems mostly correctly, but even they started to "give up" and make things up when asked for the most complex type of answer (eigenvalues).
Tier 2 (The Middle): They handled the medium problems okay but collapsed completely on the hardest ones.
Tier 3 (The Weakest): They started failing earlier and gave up very quickly, often making up answers even for medium-sized problems.

The Bottom Line

The paper concludes that these AI models don't fail randomly. Their failure is structural.

They are great at small tasks.
They struggle with medium tasks due to calculation errors.
They completely abandon the task on large, complex tasks and start making up plausible-sounding lies because their internal "working memory" runs out.

The authors released all their data and tools so other scientists can study this "working memory" limit and try to fix it. They proved that for these models, doing the math is harder than knowing the math.

Technical Summary: LinAlg-Bench

Problem Statement

Current evaluations of Large Language Models (LLMs) in mathematical reasoning often conflate difficulty with novelty, measuring only final-answer accuracy without distinguishing between knowledge deficits and execution failures. The prevailing statistical account suggests models fail due to insufficient training data or a general capability ceiling. This paper argues that LLM failure under recursive computational load is not random but structurally constrained. Specifically, it posits that failure modes shift predictably based on algorithm family and matrix dimension, transitioning from execution errors to computational abandonment at a measurable complexity threshold. Linear algebra is selected as the testbed because its operations are algorithmically precise, scale naturally via matrix dimension without altering the underlying task, and decompose into a cognitive hierarchy where each level adds exactly one computational requirement.

Methodology

The authors introduce LinAlg-Bench, a diagnostic benchmark designed to isolate computational depth from mathematical knowledge.

Dataset: The benchmark comprises 660 integer-entry problems verified against SymPy symbolic computation, distributed across 9 task types and 3 matrix dimensions ( $3\times3$ , $4\times4$ , $5\times5$ ). This results in 6,600 exhaustively evaluated model outputs.
Models: Ten frontier models were evaluated zero-shot at temperature 0, spanning diverse architectures (Dense, MoE), training paradigms (Reasoning-optimized, RLHF, Hybrid), and providers (OpenAI, Google, Anthropic, Meta, Alibaba, Mistral, DeepSeek).
Cognitive Taxonomy: Tasks are organized into five cognitive levels:
1. Reading: Pure entry extraction (Trace, Transpose).
2. Arithmetic: Parallel independent computation (Matrix-Vector, Multiplication, Power).
3. Sequential: State tracking through row operations (Rank, Nullity).
4. Recursive: Deep recursive sign management (Determinant via cofactor expansion).
5. Compositional: Built on Determinant as a sub-routine (Eigenvalues).
Forensic Pipeline: A three-stage automated "LLM-as-a-Judge" pipeline classifies 1,156 incorrect responses into ten primary error tags (e.g., SIGN_ERROR, HALLUCINATION, MEMORY_LOSS) and four eigenvalue-specific extensions. Classification follows the "first-error principle," tagging the earliest step where computation diverges from ground truth. The pipeline was validated against 593 human-annotated labels, achieving 92.6% overall agreement.
Ablation Study: A targeted study enforced algorithmically efficient strategies (Gaussian elimination) on models that naturally favored cofactor expansion to test if operation count or execution depth was the true bottleneck.

Key Results

1. The Fabrication-to-Abandonment Threshold

The central empirical finding is a sharp behavioral transition at the $4\times4$ scale.

Below Threshold ( $3\times3$ ): Models attempt computation and fail within it. Failures are dominated by execution errors such as SIGN_ERROR (33.0%) and ARITHMETIC.
Above Threshold ( $5\times5$ ): Models abandon computation before attempting it. The dominant failure mode shifts to HALLUCINATION (47.1%), specifically Complete_Collapse, where models fabricate responses through tool roleplay (simulating Python/NumPy/WolframAlpha calls they cannot access) or constraint-consistent confabulation.
Universality: This transition is near-universal across all model tiers and architectures, suggesting a working memory limit rather than a knowledge gap.

2. Tiered Performance and Failure Modes

Models separate into three distinct tiers based on their performance boundary:

Tier 1 (e.g., OpenAI-o1, Gemini-3.0-Pro, DeepSeek-V3): Maintain $\ge74\%$ accuracy on $5\times5$ determinants but converge to near-zero eigenvalue accuracy. They exhibit the "fabrication-to-abandonment" transition but sustain recursive execution longer.
Tier 2 (e.g., GPT-5.2, Mistral-Large): Partially withstand recursive loads but collapse at the compositional level.
Tier 3 (e.g., GPT-4o, Llama-3.3-70B, Claude-4.5-Sonnet): Collapse entirely at the recursive level ( $5\times5$ determinant accuracy $\le12\%$ ).

3. Constraint-Aware Confabulation

When models fail to compute eigenvalues at $5\times5$ , they do not generate arbitrary numbers. Instead, they fabricate values that satisfy partial mathematical constraints:

45% of fabricated eigenvalue sums match the matrix trace.
85% of fabricated magnitudes respect the Frobenius norm bound.
This "constraint-aware confabulation" represents a structured hallucination mode where models leverage partial knowledge to satisfy surface constraints while failing the core computation.

4. Strategy vs. Execution Depth

The study challenges the hypothesis that algorithmic efficiency (operation count) is the primary bottleneck.

Observation: In natural zero-shot settings, models rigidly locked into cofactor expansion ( $O(n!)$ ) collapsed more frequently than those using Gaussian elimination ( $O(n^3)$ ).
Ablation: When models were explicitly forced to use Gaussian elimination, accuracy did not meaningfully recover (e.g., Mistral-Large improved only to 26.9%; others remained near 0%).
Conclusion: The bottleneck is not the algorithm selected but the autoregressive execution depth. Gaussian elimination requires a strictly sequential chain of highly dependent fractional arithmetic; a single error in an early step propagates irreversibly, collapsing the result. Cofactor expansion, while mathematically more expensive, consists of parallel, independent integer steps that are safer for transformers to execute.

Significance and Claims

LinAlg-Bench contributes a forensic taxonomy and a falsifiable empirical finding regarding the structural nature of LLM mathematical failure.

Structural Constraint: LLM failure is not random but determined by algorithm family and matrix dimension. The transition from execution error to computational abandonment is a measurable threshold consistent with a working memory limit.
New Failure Mode: The paper documents constraint-aware confabulation and tool roleplay collapse as distinct, structured hallucination mechanisms that occur under computational overload, which standard accuracy-based benchmarks would misclassify as correct or fail to detect.
Mechanistic Hypotheses: The findings generate testable hypotheses for mechanistic interpretability, such as the involvement of late-layer parity circuits in sign-tracking failures and the suppression of these circuits during complete collapse.
Evaluation Methodology: The paper argues that diagnostic benchmarks should generate falsifiable mechanistic hypotheses rather than merely producing leaderboard rankings. It demonstrates that a benchmark holding the algorithm fixed while scaling dimension can isolate computational depth from mathematical knowledge.

The authors explicitly state that the fabrication-to-abandonment threshold and the working memory account are claims directly supported by the data, inviting verification through mechanistic interpretability tools and cross-domain replication. All data, model outputs, error labels, and the judge pipeline are publicly released.