Technical Summary: The Shape of Addition: Geometric Structures of Arithmetic in Large Language Models

1. Problem Statement

Despite achieving high performance on complex mathematical reasoning benchmarks (e.g., GSM8K, MATH), Large Language Models (LLMs) exhibit surprising fragility in fundamental algorithmic primitives, particularly multi-digit arithmetic. A central paradox exists: models often encode the correct information internally yet fail to output it, frequently committing "off-by-one" errors as the number of operands increases. Previous research has attempted to model these behaviors via symbolic manipulation or geometric representations, but the precise mechanism linking internal activation geometries to specific output failures (such as carry propagation errors) remains elusive. Furthermore, while lightweight probes can decode ground truth from failing models, the geometric structure allowing ground truth and hallucination to coexist within a single activation vector has not been fully elucidated.

2. Methodology

The authors investigate the fine-grained internal representations of LLMs performing 3-term addition of 10-digit integers. The study employs a multi-faceted approach combining dimensionality reduction, probing, and causal intervention:

• Dataset and Setup: Using the Qwen3-4B model, the authors constructed a dataset of 10,000 addition problems. They extract hidden state vectors ($h_p^{(L)}$) from the final layer at every generated digit position, focusing primarily on a representative middle position ($p=4$).
• Geometric Visualization: High-dimensional activation spaces are projected into 2D using UMAP (Uniform Manifold Approximation and Projection) with cosine distance. The visualization includes fixed "Anchor" vectors for digit tokens (0–9) to provide semantic reference.
• Probing Framework: Lightweight probes (logistic regression, MLPs) are trained on residual stream activations to decode various discrete and continuous variables, including Ground Truth digits, Model Output, Input Carry, Raw Sum, and a newly defined continuous variable called "Carry Potential."
• Causal Steering: To validate the geometric hypotheses, the authors define steering vectors between stable centroids on identified trajectories and intervene on activations to observe changes in generated tokens.
• Inference-Time Correction: A dual-stream consistency check method is proposed to detect and correct errors by enforcing logical alignment between decoded local (Raw Sum) and global (Carry Potential) signals.

3. Key Findings and Contributions

A. The Iso-Raw-Sum Trajectory (IRST)

The primary geometric discovery is the Iso-Raw-Sum Trajectory (IRST). The authors identify that arithmetic representations are organized into a hierarchical structure:

• Macroscopic Backbone: Representations are clustered into ten distinct "digit basins" centered around semantic anchors (0–9).
• Microscopic Texture: Within each basin, the space is stratified into parallel fibers determined by the input carry state ($c_p \in \{0, 1, 2\}$).
• Trajectory Definition: An IRST ($T_r$) is a continuous manifold connecting all internal states sharing the same raw sum ($r_p$). These trajectories act as continuous "threads" piercing through adjacent digit basins. For example, a trajectory with $r_p=1$ connects stable nodes corresponding to output digits 1, 2, and 3 as the input carry increases.

B. Arithmetic Errors as Geometric Slippages

Arithmetic errors are characterized not as random noise but as Geometric Slippages.

• Correct predictions cluster densely around stable nodes within digit basins.
• Errors occur when activation vectors drift along an IRST into the transition zones between basins or fibers.
• This drift manifests as "off-by-one" errors where the model confuses adjacent carry states (e.g., confusing $c_p=0$ with $c_p=1$), causing the representation to slide from one digit basin to an adjacent one.

C. The Noisy Quantization Model

To explain the continuity of these trajectories, the authors propose the Noisy Quantization Model:

• Continuous Carry Potential ($\Phi$): The model internally estimates a continuous "Carry Potential," representing the thrust or momentum of the carry signal derived from the context. The discrete ground truth carry is the integer floor of this potential ($c_p = \lfloor \Phi \rfloor$).
• Quantization Failure: The model's internal estimation is corrupted by neural noise ($\epsilon$). Errors occur when this noise pushes the perceived potential ($\hat{\Phi} = \Phi + \epsilon$) across an integer quantization threshold.
• Bathtub Error Distribution: This mechanism predicts a periodic "bathtub" error distribution, where error rates spike near integer boundaries (metastability) and vanish at half-integer values (robust plateaus). Empirical data confirms this pattern, allowing the estimation of the model's internal cognitive noise level ($\sigma \approx 0.05$ for 3-term addition).

D. Geometric Origins of Probe Versatility

The paper provides a geometric explanation for why lightweight probes can simultaneously decode diverse signals (Ground Truth, Model Output, Carry) from a single vector. This "probe versatility" is a direct function of the manifold's topological separability. The high separability of the Raw Sum and the structured nature of the IRST fibers allow probes to disentangle coexisting latent signals, even when the final output is erroneous.

E. Inference-Time Self-Correction

Validating the retention of correct latent signals, the authors introduce a Dual-Stream Consistency Protocol:

1. Decode the Local Calculation Stream (Raw Sum) and Global Context Stream (Carry Potential).
2. Check if the model's output is consistent with any plausible carry derived from the potential's neighborhood.
3. If inconsistent, override the output logits to project the state back onto the correct manifold fiber.
   This method significantly recovers performance (e.g., increasing token accuracy from 86.26% to 89.56% on Qwen3-4B), confirming that the model retains correct mathematical components even when the discrete token selection fails.

4. Results

• Visualization: UMAP projections reveal a clear hierarchical structure with digit basins and stratified carry fibers, consistent across different models (Qwen3-4B, Qwen3-8B, Gemma-3-4B-IT) and task complexities (3-term to 5-term addition).
• Error Modeling: The Noisy Quantization Model fits the empirical error distribution with high fidelity ($R^2 \approx 0.80$), accurately predicting the periodic nature of errors.
• Causal Validation: Steering experiments demonstrate that moving representations along the IRST direction causally changes the generated digit, with boundary-adjacent states requiring smaller perturbations to flip than stable states.
• Correction Efficacy: The dual-stream consistency check effectively corrects geometric slippages, with the majority of corrected errors being off-by-one cases. Ablation studies confirm that the local raw sum signal remains robust even when the output is wrong, isolating the noisy carry potential as the primary bottleneck.

5. Significance and Claims

The paper claims to provide a mechanistic geometric explanation for arithmetic failures in LLMs, moving beyond the view of arithmetic as purely symbolic manipulation or discrete state transitions.

• Unified Framework: It unifies the observation of probe versatility and arithmetic fragility under a single geometric framework where errors are continuous drifts on a manifold.
• Latent Signal Retention: It demonstrates that LLMs often maintain correct continuous internal representations (Carry Potential) even when they fail to produce the correct discrete output, suggesting that hallucinations in arithmetic are often quantization failures rather than a lack of knowledge.
• Universality: The IRST geometry and Noisy Quantization dynamics are observed across different model architectures and scales, suggesting a convergent mechanism for arithmetic representation in state-of-the-art LLMs.
• Practical Intervention: The work validates that inference-time interventions based on internal consistency can recover performance, offering a pathway to improve LLM reliability by stabilizing the mapping from continuous neural thoughts to discrete symbolic outputs.

The authors remain modest, noting that their current evidence combines visualization and probing rather than full circuit-level localization, and that the focus is primarily on addition. They suggest that BPE-based models with multi-digit tokens may have entangled carry signals, presenting a frontier for future research.