Arithmetic OOD Failure Unfolds in Stages in Minimal GPTs

1. Problem Statement

Arithmetic benchmarks are frequently evaluated using a single "exact match" score on held-out data. The author argues that this metric is too coarse, as it conflates qualitatively different types of failures. A model might fail to generalize from 2-digit to 3-digit addition not because it lacks the local arithmetic rules (which are often fully present in the training data), but due to specific, sequential barriers related to layout, semantics, and composition.

The core research question is: When a minimal GPT trained on exhaustive 2-digit addition fails on 3-digit cases, what specific failure occurs first, what occurs next, and what remains after targeted repairs?

2. Methodology

The study utilizes a highly controlled "sandbox" environment to isolate failure modes:

• Model Architecture: A minimal decoder-only Transformer (inspired by microGPT).
  • Analytic Stage: 1-layer, width-16, 4-head models.
  • Bridge/Residual Stage: 2-layer, width-32, 4-head models.
• Training Data: Exhaustive 2-digit addition ($100 \times 100 = 10,000$ pairs), ensuring all local digit transitions and carry rules are present in the training set.
• Evaluation Suites: A "ladder" of evaluation sets designed to introduce difficulty incrementally:
  1. Layout-only: Same 0–99 values but formatted as 3-digit strings (e.g., 049+007).
  2. True 3-digit (No Incoming Carry): Genuine 3-digit addition where the hundreds column receives no carry from the tens.
  3. True 3-digit (Incoming Carry): Hundreds column receives a carry ($c_2=1$).
  4. Thousands-carry: Results in a new thousands digit ($c_3=1$).
• Experimental Design:
  • Matched Controls: Interventions are volume-matched against controls (e.g., adding extra data vs. adding specific semantic data) to distinguish between the amount of data and the type of data.
  • Seed Sweeps: Experiments are repeated across multiple random seeds (5 or 10) to ensure robustness.
  • Metrics: Beyond exact match, the study uses teacher-forced token accuracy, logit margins, attention distributions, and conditional probabilities (e.g., $P(\text{exact} \mid \text{high2\_correct})$).

3. Key Contributions & Findings

The paper demonstrates that arithmetic Out-of-Distribution (OOD) failure is staged and can be decomposed into four distinct, experimentally separable barriers:

Stage 1: The Layout Barrier

• Observation: Models trained on 2-digit formats collapse when the same numerical values are presented in a 3-digit layout (e.g., 049 vs 49), even though the underlying arithmetic rules are identical.
• Finding: Improved positional inductive biases (e.g., coupled significance, symmetry-aware) alone do not solve this.
• Solution: Only mixed-layout exposure (training on the 3-digit rendering of the same low-range sums) materially weakens this barrier. This suggests the model relies heavily on absolute position cues rather than learning a layout-agnostic role for digits.

Stage 2: The Carry-Semantics Barrier

• Observation: Once the layout is fixed, the model treats the hundreds position not as a semantic place-value digit, but as a binary carry flag (indicating if a carry occurred from the tens column).
• Evidence:
  • Logit margins strongly favor the "flag" digit over the true digit.
  • Attention mechanisms focus on lower-order digits as indicators rather than treating the hundreds digit as an independent value.
• Solution: Targeted carry probes (specific training examples designed to teach the hundreds digit its true value) reverse the logit margin and reduce attention to lower digits. A matched control with extra generic data does not achieve this, proving the failure is semantic, not just a lack of optimization.

Stage 3: The Conditional Recomposition Barrier

• Observation: After fixing the carry semantics, the model often gets the upper digits (hundreds) correct but fails to attach the correct lower tail (tens/units).
• Finding: The bottleneck is conditional recomposition. The model knows the correct upper state but cannot bind the correct lower tail to it.
• Solution: High-conditioned tail data (training examples where the lower tail is presented in the context of the correct upper state) significantly outperforms "high-only" or "tail-only" data.
• Validation: This ordering ($\text{tail}_{\text{high}} > \text{tail} > \text{control}$) holds true in both the 1-layer analytic model and the larger 2-layer bridge model.

Stage 4: The Late Tens-Residual Barrier

• Observation: Once recomposition is largely repaired, the remaining errors are highly localized to the tens digit and are dependent on the carry state ($c_2$).
• Finding:
  • When $c_2=0$ (no carry), the model systematically under-shoots the tens digit.
  • When $c_2=1$ (carry), the model systematically over-shoots the tens digit.
• Solution: A sign-aware tens repair (specifically tenspolarity intervention) corrects these directional biases.
• Impact: On the hardest "thousands-carry" suite, this repair raises exact match from 0.664 to 0.822.

4. Significance and Implications

• Decomposition of Failure: The paper moves beyond "black box" accuracy scores, providing a vocabulary to describe why arithmetic generalization fails (layout confusion, semantic misassignment, failed binding, and sign errors).
• Methodological Shift: It argues that arithmetic benchmarks should be structured as a "ladder" of specific difficulties rather than random hold-out sets. Random splits are insufficient once local rules are saturated; structured barriers are required to reveal specific failure modes.
• Interpretability: The findings suggest that small models learn arithmetic in a specific, sequential order. Interventions must be matched to the specific active barrier (e.g., layout exposure for layout issues, semantic probes for carry issues) rather than generic data augmentation.
• Broader Context: The results imply that "brittle" arithmetic performance in larger LLMs may stem from similar layered failures (layout, semantics, composition) that are masked by aggregate scores.

Conclusion

Shintani concludes that arithmetic OOD failure is not monolithic. It unfolds in a predictable sequence: Layout $\rightarrow$ Carry Semantics $\rightarrow$ Conditional Recomposition $\rightarrow$ Tens-Residual. By isolating these stages, researchers can design targeted interventions to repair specific computational bottlenecks, offering a more rigorous framework for evaluating and improving algorithmic generalization in neural networks.