🤖 AI

QA-Merging: Query-Adaptive Reasoning via Layer Selective Model Merging

The paper introduces QA-Merging, a training-free framework that adaptively combines Long-CoT and Short-CoT models by selectively calibrating only the transformer layers with high reasoning pattern divergence, thereby reducing inference costs while maintaining strong performance across diverse reasoning tasks.

Original authors: Zhaofeng Zhong, Wei Yuan, Tong Chen, Liang Qu, Xiangyu Zhao, Quoc Viet Hung Nguyen, Hongzhi Yin

Published 2026-08-18

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhaofeng Zhong, Wei Yuan, Tong Chen, Liang Qu, Xiangyu Zhao, Quoc Viet Hung Nguyen, Hongzhi Yin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the rapidly evolving world of artificial intelligence, a specific class of computer programs known as large reasoning models has emerged as a powerful tool for solving complex problems. These systems are designed to mimic human thought processes by generating a long, step-by-step chain of reasoning before arriving at a final answer. This method, often called a "long chain of thought," allows the model to check its own work, correct errors, and navigate difficult logical puzzles with high accuracy. However, this thoroughness comes with a significant cost. When faced with a simple question that requires only a few steps, the model often continues to generate lengthy, unnecessary explanations. This behavior, sometimes described as overthinking, wastes computing power, slows down the response time, and can even introduce new errors where none existed before. The challenge for researchers has been to create a system that knows when to think deeply and when to provide a quick, direct answer, without having to rebuild the entire model from scratch every time.

A team of researchers at The University of Queensland and other institutions has addressed this dilemma with a new approach called QA-Merging. Instead of training a single model to learn how to switch between deep thinking and quick answers—a process that requires massive amounts of data and expensive computing time—they combined two existing models into one. One model was specialized in generating detailed, long chains of thought for hard problems, while the other was trained to give concise, short answers for simple tasks. By merging these two distinct "personalities" into a single entity, the researchers created a system that can adapt its behavior based on the specific question it receives. The result is a model that retains the ability to solve complex math problems with deep reasoning but can instantly switch to a brief, efficient response for easier queries, effectively eliminating the waste of overthinking.

The core of this discovery lies in how the researchers decided to combine the two models. They did not simply average the mathematical settings of the two programs, a common technique that often blurs their unique strengths. Instead, they observed that the difference between long and short reasoning is not spread evenly throughout the model's internal layers. Think of the model as a multi-layered structure where information passes through many stages. The researchers found that the divergence between the two thinking styles is concentrated in just a few specific layers, while the rest of the structure remains quite similar. By identifying these critical layers, they could focus their efforts on calibrating only those parts, leaving the others to be adjusted with a much simpler, faster method. This selective strategy allowed them to preserve the deep reasoning capabilities of the complex model while integrating the efficiency of the simple one, all without the heavy cost of retraining.

To teach the merged model how to choose the right path, the team constructed a small, carefully labeled dataset. They fed a variety of questions to both the long-thinking and short-thinking models and compared the results. If a question was difficult and only the long-thinking model solved it correctly, the merged model was instructed to follow the long-thinking pattern. If a question was simple and both models got it right, the system was guided to prefer the shorter, more efficient answer. This process created a set of rules that told the merged model exactly which reasoning style to adopt for each specific query. The researchers then applied a technique called feature alignment to the critical layers, ensuring that the merged model's internal state matched the preferred style for that particular question. For the remaining layers, they used a mathematical correction that adjusted the output without needing complex calculations, ensuring the system remained stable and efficient.

The results of this approach were striking. When tested on seven different reasoning benchmarks, ranging from grade-school math problems to advanced graduate-level science questions, the merged model consistently outperformed existing methods. On a specific 1.5-billion parameter model, the new approach reduced the average length of the response by 70 percent compared to the long-thinking model, while actually improving the overall accuracy. This means the system was not only faster and cheaper to run but also more correct. It managed to solve difficult problems with the same depth as the original complex model but avoided the wasteful overthinking on simpler tasks. In contrast, other methods that tried to achieve similar goals either required extensive retraining, which is slow and expensive, or relied on simple prompts that often failed to guide the model correctly. The new method achieved a balance that previous techniques could not, proving that it is possible to have both speed and intelligence in a single system.

The study also highlighted the importance of not treating all parts of an artificial intelligence model the same way. By showing that only a small subset of layers is responsible for the difference between deep and shallow reasoning, the researchers demonstrated that a targeted approach is far more effective than a blanket one. This insight suggests that future improvements in artificial intelligence may not always require building larger or more complex models, but rather finding smarter ways to combine and refine the capabilities of existing ones. The work provides a clear path forward for making these powerful tools more practical for everyday use, where speed and cost are just as important as accuracy. By learning to switch between modes of thought, the merged model mimics a more human-like efficiency, knowing when to pause and think deeply and when to act quickly, all without the need for expensive training or complex instructions.

The researchers verified their findings across two different sizes of models, confirming that the method works regardless of the scale of the system. They compared their results against a wide range of competitors, including models trained with reinforcement learning and those guided by specific prompts. In every case, the merged model offered a better trade-off between accuracy and efficiency. It was able to match the performance of the most thorough models on hard tasks while generating significantly fewer words, which translates directly to lower energy consumption and faster response times. The study concludes that this layer-selective merging technique is a viable and efficient alternative to the costly training methods currently used to adapt large models. It offers a practical solution to the problem of overthinking, ensuring that artificial intelligence can be both powerful and economical, ready to handle everything from simple arithmetic to complex logical deductions with the appropriate level of effort.

Technical Summary: QA-Merging

Problem Statement
Recent Large Reasoning Models (LRMs) have demonstrated strong performance on complex tasks by generating long chains of thought (Long-CoT). However, this capability often leads to "overthinking" on simple queries, introducing unnecessary intermediate reasoning steps that increase inference costs and latency without improving accuracy. Existing solutions for adaptive reasoning—dynamically selecting between Long-CoT and Short-CoT based on query difficulty—typically rely on either prompt-guided strategies or training-based methods. Prompt-guided approaches are often sensitive to prompt formulation and instruction-following capabilities, while training-based methods (e.g., Supervised Fine-Tuning or Reinforcement Learning) require large-scale datasets and incur prohibitive computational costs. Furthermore, existing model merging techniques, which offer a training-free alternative, are often static (input-agnostic) or require costly all-layer calibration, failing to provide the necessary query-level adaptivity.

Methodology
The paper proposes QA-Merging (Query-adaptive Layer Selective Merging), an activation-based framework that merges a Long-CoT model ( $\theta_L$ ) and a Short-CoT model ( $\theta_S$ ) into a single query-adaptive reasoner ( $\theta_M$ ) without retraining from scratch. The methodology consists of three core components:

Pattern-Labeled Data Construction:
The authors construct a compact calibration dataset ( $\mathcal{D}_{PL}$ ) by sampling responses from both $\theta_L$ and $\theta_S$ on seed queries. For each query, the preferred reasoning pattern is determined based on empirical correctness and response length:
- If one model is significantly more accurate, that model's pattern is preferred.
- If both are correct, the model with the shorter response is preferred.
  This process partitions the data into $\mathcal{D}_{Long}$ (queries requiring deep reasoning) and $\mathcal{D}_{Short}$ (queries solvable concisely), retaining the shortest correct responses from both models for each query.
Layer-Selective Strategy:
Motivated by empirical analysis showing that reasoning pattern divergence (RPD) between $\theta_L$ and $\theta_S$ is non-uniform across transformer layers, QA-Merging avoids calibrating all layers.
- High-Divergence Layers ( $L_{imp}$ ): The top- $K$ layers with the highest RPD (identified via hidden-state cosine distance) are selected for intensive calibration.
- Low-Divergence Layers: The remaining layers are handled via a lightweight closed-form correction.
Merging Mechanisms:
- Feature-Alignment Calibration with Contrastive Shaping (for $L_{imp}$ ): For the important layers, element-wise merging coefficients ( $\lambda_L, \lambda_S$ ) are learned to minimize the distance between the merged model's hidden states and the preferred model's hidden states (positive model), while maximizing distance from the non-preferred model (negative model). This is achieved via a joint objective combining $L_2$ feature alignment and a contrastive loss.
- Closed-Form Hidden-State Correction (for $L \notin L_{imp}$ ): For the remaining layers, the model is initialized via average parameter merging. To correct residual hidden-state distortions, a linear transformation matrix is computed in closed form to map the merged hidden states toward the specialist models' representations. A preference-weighted correction matrix is then applied to balance Long-CoT and Short-CoT behaviors.

Key Contributions

QA-Merging Framework: A novel model merging approach that enables a single merged model to dynamically switch between Long-CoT and Short-CoT reasoning patterns based on query difficulty, avoiding expensive retraining.
Layer-Selective Calibration: The identification that reasoning divergence is concentrated in a subset of transformer layers. The proposed strategy calibrates only these high-divergence layers while applying efficient closed-form corrections to the rest, significantly improving merging efficiency.
Training Data Construction: A strategy to build a compact, pattern-labeled dataset that provides query-level supervision for determining the optimal reasoning pattern for each input.

Experimental Results
The authors evaluated QA-Merging on seven reasoning benchmarks (including GSM8K, MATH500, AIME24/25, and GPQA) across two model scales (1.5B and 4B parameters).

Performance vs. Efficiency: QA-Merging achieved a superior accuracy-efficiency trade-off compared to prompt-guided methods, training-based adaptive reasoning (e.g., Ada-R1, HAPO), and existing merging baselines (arithmetic and activation-based).
Specific Gains: On the Qwen2.5-1.5B model pair, QA-Merging reduced the average response length by 70% (from ~8162 to ~2404 tokens) while improving average accuracy by 0.6 percentage points over the Long-CoT base model.
Ablation Studies: Removing the layer-selective strategy or the feature-alignment calibration significantly degraded performance, confirming the necessity of targeting high-divergence layers and explicitly aligning representations.
Efficiency: The merging process required only ~20 minutes on a single GPU using a small calibration set (128 queries), contrasting sharply with the 8–24+ hours required for training-based RL methods.

Significance
The paper claims that QA-Merging offers a highly efficient and effective alternative to costly training methods for achieving adaptive reasoning. By leveraging the observation that reasoning patterns diverge unevenly across layers, the method successfully integrates the deep reasoning capabilities of Long-CoT models with the efficiency of Short-CoT models. The results suggest that query-adaptive model merging can match or surpass the performance of training-based approaches while drastically reducing computational overhead, making adaptive reasoning more accessible and practical for deployment.

Technical Summary: QA-Merging

More like this