Technical Summary: Reasoning-Trace Collapse in Fine-Tuned Models

Problem Definition

The paper identifies a structural failure mode termed reasoning-trace collapse, which occurs when explicit reasoning models are fine-tuned on standard instruction–response datasets that lack intermediate reasoning traces. While these models are initially trained to produce structured reasoning steps (e.g., within <thought>...</thought> delimiters) before generating a final answer, downstream adaptation often uses data containing only the final answer.

The core issue is that standard supervised fine-tuning (SFT) minimizes loss by learning to produce the target answer. In the absence of reasoning traces in the training data, the model learns to treat the absence of reasoning as the desired behavior. Consequently, the model may retain high final-answer accuracy while losing the ability to generate complete, non-empty, structurally valid reasoning traces. This failure is often invisible to standard evaluation metrics (e.g., pass@1), which measure only the correctness of the final output, thereby obscuring the loss of the explicit reasoning behavior that distinguishes these models.

Methodology

Structural Evaluation Framework

To quantify this phenomenon, the authors propose a framework that decouples answer correctness from reasoning-trace validity. They define four structural categories for reasoning traces:

1. Valid: A complete, non-empty reasoning trace that can be reliably separated from the final answer.
2. Empty: Reasoning delimiters are present but contain no content.
3. Missing: No reasoning trace or delimiters are extracted.
4. Truncated: A reasoning trace begins but is incomplete (e.g., unclosed delimiters or generation limits).

The framework introduces specific metrics:

• Valid Reasoning Rate (VR): The frequency of valid traces.
• Reasoning-Conditioned Pass@1 (Rpass@1): Task accuracy computed only over generations containing valid reasoning traces. This metric helps distinguish between models that fail because their reasoning is ineffective versus models that fail because they stop producing valid reasoning.

Tooling: THINKPACK

The authors release THINKPACK, a lightweight Python library designed to operationalize this framework. It provides model-agnostic utilities for:

• Constructing prompts with reasoning-aware chat templates.
• Parsing raw model outputs into structured reasoning and answer segments.
• Computing structural reliability metrics (VR, ER, MR, TR).
• Applying loss masking strategies during training to exclude specific regions (like empty reasoning blocks) from the loss calculation.

Empirical Study

The study evaluates four open-weight reasoning models (Qwen3-8B, Olmo-3-7B, Llama-R1-8B, Nemotron-7B) fine-tuned on a Chemistry dataset (SciKnowEval) without explicit reasoning traces. The evaluation covers in-domain (Chemistry) and out-of-domain (GSM8K for math, EvalPlus for code) tasks.

The experiments compare:

1. Format Variations: Whether missing reasoning is represented as an empty <thought> block (empty-think) or omitted entirely (no-think).
2. Mitigation Strategies:
   • Standard SFT: No special handling.
   • Masked-think: The empty reasoning region is masked from the loss, preventing the model from being penalized for not generating content there, while still training on the answer.
   • Response-only: Both prompt and reasoning regions are masked; only the final answer contributes to the loss.
   • Distillation: A teacher model (GPT-5-mini) generates reasoning traces for the training data, which are then used for fine-tuning.

Key Results

1. Prevalence of Collapse

Standard fine-tuning rapidly induces reasoning-trace collapse across all models.

• Accuracy vs. Reasoning: In several settings, final-answer performance (pass@1) remains high or even improves, while the Valid Reasoning Rate (VR) drops sharply (e.g., to 0% in some cases).
• Format Dependence: The representation of missing reasoning significantly impacts collapse severity. For instance, no-think formats caused Llama-R1-8B and Olmo-3-7B to collapse more severely than empty-think formats, though Qwen3-8B showed the opposite trend. This indicates that model defaults are not reliably protective.
• Failure Modes: Different models exhibit different collapse patterns. Qwen3-8B, Llama-R1-8B, and Nemotron-7B primarily collapsed into empty or missing traces, whereas Olmo-3-7B collapsed primarily into truncated traces.

2. Effectiveness of Mitigation

• Loss Masking: Simple masking strategies (specifically masked-think) substantially preserved valid reasoning behavior without requiring teacher-generated traces. For models like Qwen3-8B and Nemotron-7B, masking maintained high VR on out-of-domain tasks while allowing adaptation to the new task.
• Distillation: While distillation was highly effective for Qwen3-8B and Llama-R1-8B in preserving reasoning structure, it performed poorly for Olmo-3-7B, where VR dropped significantly despite the presence of teacher traces. This suggests that teacher-generated traces are not a universal solution and that mitigation effectiveness is model-dependent.
• Trade-offs: Masking allows for adaptation while retaining structural reasoning, but it does not guarantee the quality or faithfulness of the reasoning, only its structural presence.

Significance and Claims

The paper argues that the evaluation of fine-tuned reasoning models is currently incomplete if it relies solely on final-answer metrics. The authors claim:

1. Reasoning-trace collapse is a distinct structural failure mode that can occur even when task accuracy is preserved.
2. Standard answer-only evaluation is insufficient for understanding whether explicit reasoning behavior survives adaptation.
3. Lightweight interventions, such as loss masking, can effectively mitigate this collapse without the cost and complexity of teacher distillation.
4. Evaluation protocols must evolve to report structural reasoning reliability metrics (VR, ER, MR, TR) alongside final-answer performance, particularly when adaptation data lacks explicit reasoning traces.

The authors conclude that while final-answer metrics remain essential, they must be supplemented with structural reliability metrics to ensure that reasoning models retain their defining characteristic: the ability to produce explicit, structured reasoning traces.