DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning Models

1. Problem Statement

Large Reasoning Models (LRMs), such as OpenAI's o1 and DeepSeek-R1, excel at complex tasks by generating long Chain-of-Thought (CoT) reasoning trajectories. However, they suffer from "overthinking," producing excessively verbose outputs with redundant steps, unnecessary backtracking, and repetitive loops. This leads to:

• High Inference Costs: Significant token usage and latency.
• Accuracy Degradation: Redundant reasoning can misguide the model toward incorrect conclusions.
• Distillation Mismatch: When distilling knowledge from a "Short-CoT" teacher (concise, polished) to a "Long-CoT" student (verbose, reflective), a learnability gap arises. The student struggles to imitate the teacher's concise style directly, often resulting in poor performance or accuracy drops.

Existing solutions like inference-time pruning (stopping generation early) risk halting reasoning prematurely, while standard distillation often fails to bridge the structural gap between verbose students and concise teachers.

2. Methodology: Distilled Reasoning Pruning (DRP)

DRP is a hybrid framework that unifies inference-time pruning with supervised fine-tuning (SFT) distillation. Instead of simply training the student on the teacher's final answer or a short CoT, DRP uses the teacher to prune and refine the student's own verbose reasoning traces.

The process consists of three stages (illustrated in Figure 2 of the paper):

A. Skill-Based Step Decomposition

• The student model generates an initial, verbose CoT trace ($T$).
• A Teacher LLM (e.g., GPT-4o) decomposes this trace into non-overlapping segments.
• Key Innovation: Unlike naive sentence splitting, the teacher assigns a functional skill label to each step (e.g., "Reading given quantity," "Algebraic representation," "Logical inference"). This ensures steps are semantically coherent and functionally aligned.

B. Step-Level Pruning and Rewriting

The teacher model evaluates each skill-labeled step and applies one of four actions:

1. Keep: Retain the step unchanged.
2. Delete: Remove redundant, uninformative, or backtracking steps.
3. Rewrite: Condense the step while preserving logical intent.
4. Merge: Combine adjacent steps into a single atomic unit.

• The teacher then reconstructs a fluent, concise reasoning trace ($\hat{T}$) that maintains the student's original tone but removes verbosity. It also updates the final answer summary ($\hat{A}$) if necessary.

C. Supervised Fine-Tuning (SFT)

• The student model is fine-tuned on pairs of $(Input, \hat{R})$, where $\hat{R}$ is the pruned, teacher-refined response.
• The objective is to minimize the negative log-likelihood of the refined trace, teaching the student to internalize skill-aligned, token-efficient reasoning strategies without losing the structural depth required for complex problems.

3. Key Contributions

1. DRP Framework: A novel method that bridges the "learnability gap" by pruning the student's reasoning path using a teacher, rather than forcing the student to mimic a teacher's completely different reasoning style.
2. Skill-Aware Decomposition: Introduces a mechanism to segment reasoning into semantically coherent, functionally labeled units. This provides a stable foundation for precise pruning, outperforming naive splitting.
3. Structural Consistency Insight: Demonstrates that effective training CoTs must be both informative and structurally consistent with the student's reasoning process. Preserving the step-wise structure (even if shortened) is more effective than naive compression.

4. Experimental Results

The authors evaluated DRP on DeepSeek-R1-Distill-Qwen models (1.5B and 7B) across in-domain (GSM8K) and out-of-distribution (OOD) benchmarks (MATH500, AIME24, AMC23).

Key Findings:

• Token Efficiency: DRP significantly reduced token usage without sacrificing accuracy.
  • GSM8K (7B): Tokens reduced from 917 to 328 (-64%), while accuracy improved from 91.7% to 94.1%.
  • AIME24 (7B): Tokens reduced by 43% with no drop in accuracy (15/30 correct).
  • AMC (1.5B): Tokens reduced from 4574 to 3657, with accuracy jumping from 22/40 to 27/40.
• Comparison with Baselines:
  • vs. Prompt-based (TALE): DRP outperformed TALE on complex OOD tasks where TALE's performance dropped significantly.
  • vs. SFT-based (CoT-Valve): DRP achieved better accuracy and compression. CoT-Valve often suffered accuracy trade-offs when compressing.
  • vs. RL-based (ThinkPrune): While ThinkPrune compressed well, DRP achieved higher accuracy, likely due to high-quality teacher guidance.
• Ablation Studies:
  • Skill Decomposition: Removing skill labels (default splitting) reduced performance, confirming that semantic alignment is crucial.
  • Direct Distillation: Training directly on short teacher CoTs (without pruning the student's trace) led to sharp accuracy drops on OOD tasks, proving that structural preservation is vital.
  • Teacher Robustness: The method worked effectively with various teachers (GPT-4o, Gemini 2.0, DeepSeek-V3), though GPT-4o yielded the best results.

5. Significance and Conclusion

• Efficiency vs. Accuracy Trade-off: DRP challenges the notion that efficiency must come at the cost of accuracy. By refining the reasoning process rather than just shortening the output, it achieves both.
• Scalability: The method is particularly effective for smaller models (1.5B), compensating for limited capacity through high-quality, structured supervision.
• Robustness: It mitigates "degenerate loops" (where models get stuck generating infinite text), leading to more stable inference.
• Future Impact: This work provides a blueprint for training efficient reasoning models that can handle complex, long-horizon tasks without the computational overhead of "overthinking," making advanced reasoning more accessible and cost-effective.

In summary, DRP solves the inefficiency of Large Reasoning Models by using a teacher to surgically remove redundancy from a student's own reasoning path, guided by functional skills, resulting in models that are faster, cheaper, and often more accurate.