ArithmAttack: Evaluating Robustness of LLMs to Noisy Context in Math Problem Solving

1. Problem Statement

While Large Language Models (LLMs) have demonstrated significant progress in mathematical reasoning, their robustness against adversarial noise remains under-explored. Previous studies have focused on semantic perturbations (e.g., synonym replacement) or typographical errors, which often alter the meaning or features of the input.

The authors identify a gap in understanding how LLMs handle syntactic noise that preserves semantic meaning but disrupts the token flow. Specifically, the paper investigates whether inserting random punctuation marks into math problem prompts—without adding, deleting, or changing any words—can degrade the model's reasoning capabilities. The core research question is: How do LLMs respond to noise attacks consisting of random punctuation marks in the context of math problem-solving?

2. Methodology

ArithmAttack Framework

Inspired by the AEDA (An Easier Data Augmentation) method, the authors propose ArithmAttack, a noise injection technique designed to test robustness while maintaining semantic integrity.

• Noise Mechanism: Random punctuation marks (., ,, !, ?, ;, :) are inserted into the input text.
• Semantic Preservation: No words are added, deleted, or modified. Consequently, the semantic similarity between the original and perturbed sentences remains 100% (verified via Universal Sentence Encoder).
• Noise Levels: The attack is applied at three distinct intensities, corresponding to 10%, 30%, and 50% of the sentence length (based on word count).

Experimental Setup

• Datasets:
  • GSM8K: 1,320 test samples of grade-school math word problems (high linguistic diversity).
  • MultiArith: 180 test samples covering various arithmetic complexities.
• Models Evaluated: Eight LLMs with varying parameter counts (1.5B to 8B) and training origins:
  • Llama Family: Llama-3-8B, Llama-3.1-8B.
  • Mistral Family: Mistral-7B, Mathstral-7B (math-specialized).
  • Others: Gemma2-2B, Zephyr-7B, Qwen2.5-1.5B, DeepSeek-R1-Distill-Llama-8B.
• Prompting Strategy: Zero-Shot Chain-of-Thought (CoT) prompting was used to elicit step-by-step reasoning.
• Evaluation Metrics:
  • Accuracy: Percentage of correctly solved problems.
  • Attack Success Rate (ASR): The ratio of answers that were correct in the clean setting but became incorrect after the noise attack.
  • Miss Rate: The rate at which the model fails to extract the final answer in the expected format.

3. Key Contributions

1. Proposal of ArithmAttack: A novel, lightweight attack vector that introduces random punctuation to math prompts. It is distinct from previous attacks because it causes zero information loss and maintains 100% semantic similarity, isolating the impact of syntactic disruption on reasoning.
2. Comprehensive Robustness Evaluation: The study evaluates eight diverse LLMs across two standard math datasets, providing a comparative analysis of how different architectures and training data (general vs. math-specialized) handle syntactic noise.
3. Empirical Evidence of Vulnerability: The paper demonstrates that even high-performing models are highly susceptible to punctuation noise, with performance degrading linearly as noise intensity increases.

4. Key Results

Performance Degradation

• Universal Vulnerability: All eight models showed a decline in accuracy as the percentage of punctuation noise increased from 10% to 50%.
• Dataset Sensitivity: The performance drop was more pronounced on the GSM8K dataset (more complex problems) compared to MultiArith, suggesting that models struggle more with noise when the underlying reasoning task is difficult.

Model-Specific Findings

• Most Robust: Llama-3.1-8B demonstrated the highest overall robustness, maintaining the best accuracy across both datasets and noise levels. It also had the lowest Attack Success Rate (ASR) on GSM8K.
• Math-Specialized Advantage: Mathstral outperformed its base counterpart, Mistral, in robustness. This suggests that specialized mathematical training provides some resilience against syntactic noise.
• Least Robust: Zephyr exhibited the poorest performance, with the lowest clean accuracy and the highest ASR (up to ~77% on MultiArith), indicating a severe lack of robustness.
• DeepSeek: Showed comparable robustness to Llama models on the MultiArith dataset but slightly lower on GSM8K.

Attack Success Rate (ASR)

• ASR increased significantly with noise levels. For example, on GSM8K, Zephyr's ASR reached nearly 75%, meaning three out of four previously correct answers became incorrect due to punctuation noise.
• Llama models consistently maintained the lowest ASR, indicating they are less likely to be "fooled" by this specific type of attack.

Answer Extraction

• The "Miss Rate" (failure to extract the final answer) was generally low (<4%) for most models, except for Zephyr on MultiArith (12.8%). This indicates that the primary failure mode is reasoning breakdown rather than formatting failure.

5. Significance and Future Directions

• Vulnerability Identification: The study reveals that LLMs are surprisingly fragile to simple syntactic perturbations that do not alter meaning. This challenges the assumption that semantic preservation guarantees robustness.
• Implications for Safety: Since punctuation noise is easy to generate and hard to detect semantically, it represents a potential vector for adversarial attacks in real-world applications where input formatting might be corrupted or manipulated.
• Future Work: The authors suggest extending this research to:
  • Other types of noise (grammatical errors, wrong abbreviations).
  • Different reasoning tasks (logical and causal reasoning).
  • Investigating the mechanism of why punctuation disrupts the reasoning flow (e.g., attention mechanism shifts).

In conclusion, ArithmAttack establishes that current LLMs, even those fine-tuned for math, lack robustness against non-semantic syntactic noise, highlighting a critical area for improvement in model training and evaluation.