Technical Summary: Beyond Correctness: Benchmarking and Aligning Response Behaviors in Hybrid-Thinking MLLMs

1. Problem Statement

Multimodal Large Language Models (MLLMs) are increasingly deployed with hybrid-thinking interfaces, which offer two distinct inference modes: a deliberative thinking mode (allocating additional test-time computation for reasoning) and a non-thinking mode (returning direct answers under strict latency and token budgets). While these modes differ in reasoning budget, they are expected to satisfy the same user-facing quality standards.

The paper identifies a critical gap: Correctness alone is insufficient to characterize response quality. A model may produce a factually correct answer in non-thinking mode while exhibiting undesirable response behaviors, such as leaking internal reasoning traces, repeating content, making logical contradictions, or engaging in performative reasoning without information gain. The authors term the discrepancy in these user-visible behaviors between the two modes response-pattern misalignment. Current optimization strategies often prioritize answer correctness or efficiency, failing to constrain the form of the delivered response, leading to systematic misalignment where non-thinking interfaces exhibit significantly higher failure rates than their thinking counterparts.

2. Methodology

2.1 PatternEval: A Failure-Enriched Diagnostic Benchmark

To quantify this misalignment, the authors introduce PatternEval, a benchmark comprising 2,415 multimodal prompts spanning three task families:

1. Visual Perception and Grounding (VG): Includes OOD perception, content recognition, grounding, and factuality.
2. OCR and Structured-Image Understanding (OS): Includes OCR and chart understanding.
3. Multimodal Knowledge Reasoning (KR): Includes STEM, knowledge, and general reasoning tasks.

PatternEval is specifically constructed to stress-test four recurrent response-pattern failures:

• Chain-of-Thought (CoT) Leakage: Internal process traces, draft analysis, or thinking markers appearing in the final user-visible answer.
• Response Repetition (Rep): Unnecessary repetition of sentences, semantic content, or structure.
• Logical Contradiction (Con): Mutually incompatible claims within the same response.
• Performative Reasoning (PR): Unsupported reasoning wrappers or analysis that yields no information gain.

Evaluation Protocol:

• Judges: A verifier judge (Qwen3-Max) assesses task correctness ($c$), while a pattern judge (Seed-2.0-Pro with image access) assigns binary labels for the four failure types ($b_{cot}, b_{rep}, b_{con}, b_{pr}$).
• Metrics: The benchmark reports Accuracy ($Acc$) and Trigger Rate ($Trigger$), defined as the proportion of responses exhibiting at least one failure pattern. The gap between modes is measured as $\Delta_{pat} = Trigger_{NT} - Trigger_{T}$.

2.2 PatternRM and PatternRL: Alignment via Reinforcement Learning

To address the identified misalignment, the authors propose a post-training framework:

• PatternRM (Reward Model): A specialized reward model trained via Supervised Fine-Tuning (SFT) on 57,578 annotated instances. It predicts the four failure labels based on the generated text (text-only input for efficiency). It is trained to approximate the human-aligned pattern judge.
• PatternRL (Reinforcement Learning): A policy optimization method that incorporates pattern-specific penalties into the reward function.
  • Reward Design: The final reward $r$ combines a verifier reward for correctness ($s_{ver}$) and an auxiliary pattern reward ($s_{pat}$).
  • $s_{pat} = -z \cdot \min(0.1, \sum w_p \hat{b}^p_{RM})$, where $w$ are weights for each failure type (higher for CoT and Rep, lower for Con and PR).
  • This design ensures that incorrect answers receive zero reward, while correct answers are penalized if they exhibit undesirable patterns, without overriding the primary correctness objective.

3. Key Results

3.1 Benchmark Findings (PatternEval)

Evaluation across 25 model configurations (including Qwen, Kimi, Seed, Claude, Mimo, and GPT families) reveals:

• Systematic Misalignment: Non-thinking inference consistently exhibits higher failure rates than thinking inference. The gap ($\Delta_{pat}$) is positive for all models, with some frontier models showing gaps as high as 48.64 percentage points (e.g., Kimi-K2.6).
• Dominant Failure Modes: CoT leakage and response repetition are the most prevalent failures, particularly in non-thinking mode.
• Capability vs. Alignment: Increasing model scale (e.g., from 4B to 397B in the Qwen3.5 series) improves task accuracy but does not uniformly reduce the cross-mode Trigger gap. The gap remains substantial (21–26 percentage points) even in larger models.
• Length Correlation: There is a strong positive correlation between response length and Trigger rates ($r=0.84$ in thinking mode, $r=0.64$ in non-thinking mode). Incorrect responses are generally longer and more failure-prone than correct ones.

3.2 Post-Training Outcomes (PatternRL)

Experiments on Qwen3-VL-4B and Qwen3-VL-8B demonstrate:

• Reduction in Failures: PatternRL significantly reduces the non-thinking Trigger rate compared to a correctness-only baseline (BaseRL).
  • Qwen3-VL-4B: Trigger reduced by 13.08 percentage points.
  • Qwen3-VL-8B: Trigger reduced by 14.35 percentage points.
• Accuracy Trade-off: While PatternRL preserves aggregate accuracy relative to the baseline in the 8B model, it incurs a marginal trade-off in task performance, particularly for the 4B model where accuracy declines slightly across math and logic tasks.
• Generalization: The method effectively mitigates response-pattern failures across diverse task families (VG, OS, KR) without requiring separate models for different modes.

4. Contributions

1. Formulation of Response-Pattern Alignment: The paper defines response-pattern alignment as a requirement complementary to answer correctness, highlighting that user-visible behavior must remain consistent across inference modes.
2. PatternEval Benchmark: Introduction of a failure-enriched diagnostic set that stress-tests four specific user-visible failures under hybrid-thinking interfaces, revealing a consistent misalignment gap across state-of-the-art models.
3. PatternRM and PatternRL Framework: Development of a response-level reward model and a reinforcement learning strategy that explicitly penalizes response-pattern failures. This approach successfully mitigates misalignment while largely preserving task performance.

5. Significance and Claims

The paper argues that controllable reasoning effort should not come at the cost of unstable response behavior. It demonstrates that strong task performance does not guarantee stable response patterns across different inference interfaces. By introducing PatternEval and PatternRL, the authors provide a practical framework for evaluating and training hybrid-thinking MLLMs to ensure that non-thinking modes deliver responses that are not only correct but also coherent, concise, and free of unintended process narration.

The authors modestly note that while PatternRL reduces failures, it does not eliminate them entirely, suggesting that fundamental improvements may require higher-quality supervision data and constraints applied during earlier training stages (e.g., SFT) rather than solely during RL. Additionally, they observe a capacity-dependent trade-off, where smaller models may suffer more from the constraints imposed by pattern-aware optimization.