Red Teaming Large Reasoning Models

1. Problem Statement

Large Reasoning Models (LRMs) represent a significant evolution from standard Large Language Models (LLMs) by generating explicit, traceable Chains of Thought (CoT) to solve complex, multi-step tasks. While this enhances interpretability and logical consistency, the paper argues that the reliance on intermediate reasoning steps introduces novel safety and reliability risks not present in traditional LLMs.

Existing evaluation benchmarks fail to capture these specific vulnerabilities because they:

• Focus on isolated failure modes (e.g., only jailbreaking or only factual accuracy).
• Lack a unified framework to compare LRMs against their base LLM counterparts.
• Do not account for reasoning-specific risks such as:
  • CoT-hijacking: Adversaries manipulating the reasoning chain to force untruthful or unsafe outputs.
  • Prompt-induced inefficiencies: Triggers that cause "overthinking," leading to unnecessary token generation, high latency, and computational waste.

2. Methodology: Rt-LRM Benchmark

The authors propose Rt-LRM, a unified benchmark designed to assess the trustworthiness of LRMs across three core dimensions: Truthfulness, Safety, and Efficiency.

A. Evaluation Dimensions & Tasks

The benchmark consists of 30 carefully curated tasks (11 CoT-hijacking risks and 18 prompt-induced impacts) covering diverse scenarios:

1. Truthfulness:
   • Objective Truth: Proportional operations, compositional calculations, and contextualized problem solving.
   • Conceptual Truth: Handling controversial issues, stereotypes, misconceptions, fictional content, and conspiracy theories.
2. Safety:
   • Societal Safety: Economic crime, copyright violations, chemical/biological threats, cybercrime, and misinformation.
   • Personal Safety: Violence, self-harm, sexual crime, and harassment.
   • Attack Vectors: Uses CoT-hijacking (injecting errors into reasoning steps) and Prompt-induced impacts (embedding jailbreaks or distractions).
3. Efficiency:
   • Measures Overthinking Rate (OR) and Reasoning Time.
   • Tasks include mathematical QA, symbolic reasoning, code generation, and recursive reasoning designed to induce loops or excessive token usage.

B. Evaluation Metrics

• Accuracy (Acc): For truthfulness (fact-checking).
• Attack Success Rate (ASR) & Toxicity Score: For safety (measuring the proportion of successful adversarial attacks and the severity of harmful content).
• Overthinking Rate (OR): A token-based metric defined as the proportion of samples where the token count of a triggered input exceeds 2x the token count of the clean input. This normalizes for model-specific generation styles.
• Reasoning Time: Supplementary metric for latency.

C. Experimental Setup

• Models Evaluated: 26 state-of-the-art models, including both Open-Source (e.g., Qwen, GLM, LLaMA, DeepSeek variants) and Proprietary (e.g., o1, o3-mini, Gemini, Claude) models.
• Comparison: Each LRM is compared against its corresponding base LLM (without reasoning capabilities) to isolate reasoning-specific vulnerabilities.
• Toolbox: A scalable, modular Python toolbox is released to standardize dataset configuration, reasoning logic, and metric computation.

3. Key Contributions

1. Rt-LRM Benchmark: The first unified framework to systematically evaluate LRMs across truthfulness, safety, and efficiency, specifically targeting reasoning-induced vulnerabilities.
2. Unified Attack Framework: Introduces a dual-perspective attack strategy combining CoT-hijacking (direct reasoning manipulation) and Prompt-induced impacts (indirect distractions/overthinking).
3. Scalable Toolbox: Provides a reproducible, modular infrastructure for future trustworthiness research in LRMs.
4. Comprehensive Empirical Analysis: Extensive evaluation of 26 models revealing critical insights into the trade-offs of explicit reasoning.

4. Key Results & Findings

The experiments yielded six major "Takeaways":

• Takeaway 1: LRMs are generally less trustworthy than base LLMs.

  • Despite enhanced reasoning, LRMs consistently show lower truthfulness, higher Attack Success Rates (ASR), and higher Overthinking Rates (OR) compared to their base versions.
  • Example: GLM-4-Z1-32B had an ASR of 70.06% vs. 53.84% for its base model; DPSK-Qwen-32B had an OR of 78.50% vs. 56.50%.
  • Implication: Explicit reasoning mechanisms create new attack surfaces.

• Takeaway 2: Widespread trustworthiness challenges exist, but proprietary models perform better.

  • Open-source models struggle significantly (ASR > 50% for many).
  • Proprietary models (e.g., Claude-Sonnet-4, o1) show relative superiority in safety and efficiency but still exhibit critical vulnerabilities.

• Takeaway 3: Truthfulness declines with task complexity.

  • Models perform reasonably well on simple math but degrade significantly on context-dependent or abstract tasks (e.g., handling stereotypes or conspiracy theories), suggesting reliance on superficial patterns rather than deep reasoning.

• Takeaway 4: Persistent safety risks across societal and personal contexts.

  • Safety failures are not isolated to specific training paradigms. Some models (e.g., MiMo-RL) showed extremely high vulnerability to self-harm prompts (97.06% ASR).

• Takeaway 5: Systemic inefficiency (Overthinking).

  • LRMs frequently enter "overthinking" loops, generating excessive tokens even on simple prompts.
  • Example: GLM-4-Z1-32B exhibited OR > 70% across all efficiency tasks.

• Takeaway 6: Training strategy correlations.

  • SFT+RL (Supervised Fine-Tuning + Reinforcement Learning) models generally offer the best balance of truthfulness, safety, and efficiency.
  • RL-only models tend to have lower overthinking but suffer in truthfulness and safety.
  • SFT-only models often show the worst overall performance in this benchmark.

5. Significance

• Paradigm Shift: Challenges the prevailing assumption that "stronger reasoning" universally leads to "more trustworthy" models. The paper demonstrates that the reasoning process itself can be a liability.
• New Vulnerability Surface: Highlights that the intermediate CoT steps are a primary vector for attacks (hijacking) and inefficiency (overthinking), distinct from standard LLM prompt injection.
• Future Directions: Urges the development of targeted defenses, such as:
  • Training-time alignment with safety signals in reasoning chains.
  • Inference-time monitors to detect overthinking or logical loops.
  • External guard models to filter unsafe reasoning paths.
• Community Resource: The release of Rt-LRM and its toolbox provides a standardized foundation for the community to develop and evaluate safer, more reliable reasoning systems.

In conclusion, the paper establishes that while LRMs are powerful, they currently suffer from a "trustworthiness gap" compared to base LLMs, necessitating a new generation of evaluation benchmarks and defense mechanisms tailored specifically to the reasoning process.