Adaptive Multi-Expert Reasoning via Difficulty-Aware Routing and Uncertainty-Guided Aggregation

The paper introduces Adaptive Multi-Expert Reasoning (AMR), a framework that enhances math reasoning robustness by dynamically routing problems to specialized experts based on predicted difficulty and uncertainty, ultimately achieving superior accuracy on GSM8K compared to similarly sized models trained on synthetic data.

The Gist

Imagine you have a team of three brilliant but very different math tutors trying to solve a tricky homework problem for you.

• Tutor A loves writing out long, strict equations.
• Tutor B is great at doing mental math and explaining things in plain English.
• Tutor C is a perfectionist who breaks every problem down into tiny, step-by-step instructions.

In the past, if you asked a computer (a Large Language Model) to solve a math problem, it would just pick one of these tutors, try to solve it, and hope for the best. If the problem was hard, the tutor might get confused and give a wrong answer.

This paper introduces a new system called AMR (Adaptive Multi-Expert Reasoning). Think of AMR not as a single tutor, but as a smart project manager who oversees these three experts. Here is how it works, broken down into simple steps:

1. The "Difficulty Detector" (The Router)

Before the experts even start working, the Project Manager looks at the problem and asks: "How hard is this?" and "How unsure am I about the answer?"

• If the problem is easy (like "2 + 2"), the manager says, "Okay, just one expert can handle this quickly."
• If the problem is medium, the manager says, "Let's get two experts to try it just to be safe."
• If the problem is really hard (like a complex word problem), the manager says, "This is tricky! Let's get all three experts to try it, and let's have them try a few different ways to solve it."

This is called Difficulty-Aware Routing. Instead of treating every problem the same, the system adapts its effort based on how hard the task is.

2. The "Drafting & Editing" Phase (Correction & Finalization)

Once the experts generate their answers, they aren't perfect yet.

• Correction Pass: The system takes the best draft and asks the "Step-by-Step" expert to look for mistakes and fix them, just like a teacher correcting a student's homework.
• Finalization Pass: The system then asks for a clean, polished version of the answer that is easy to read.

3. The "Referee" (Neural Verifier)

Now you have several different answers. How do you know which one is right?
Enter the Referee. This is a special AI trained specifically to spot the correct answer. It looks at all the drafts and gives each one a "confidence score" (e.g., "I'm 90% sure this answer is correct").

4. The "Group Vote" (Clustering Aggregation)

Finally, the system groups the answers. If three different experts all came up with the number "42," that's a strong signal.
The system uses a special formula that combines:

• How much the Referee trusts the answer.
• How well-structured the answer is.
• How many experts agreed on that specific number.

The answer with the highest combined score wins.

Why is this a big deal?

Most other smart math models try to get better by eating more data. They are trained on millions of fake math problems created by other computers (synthetic data) to "memorize" how to solve things.

AMR is different. It didn't eat any extra data. It only used the original, standard math problems it was supposed to learn. Yet, it scored 75.28% on a tough test (GSM8K).

The Analogy:
Imagine two students taking a test:

• Student A (The old way) memorized 10,000 practice questions. They are good, but if the test asks a question slightly differently, they get confused.
• Student B (AMR) only studied the 100 official practice questions. But, when they see a hard question, they know to call a friend for help, double-check their work, and ask a teacher to verify the answer before writing it down.

The Result: Student B (AMR) performed better than almost all the other 7-billion-parameter models, even those that had memorized massive amounts of extra data.

The Takeaway

This paper proves that you don't always need a bigger brain or more data to be smarter. Sometimes, you just need a better strategy. By knowing when to work hard, when to ask for help, and how to double-check your work, a computer can solve math problems much more reliably and efficiently.

Technical Summary

1. Problem Statement

Large Language Models (LLMs) have demonstrated strong capabilities in mathematical reasoning (e.g., on the GSM8K benchmark) but suffer from inconsistent performance across problems of varying difficulty. Current approaches face two main limitations:

• Lack of Flexibility: Static prompting or uniform ensemble averaging fails to adapt to the specific complexity or reasoning style required by a given problem.
• Data Inefficiency: State-of-the-art performance often relies on training on massive synthetic datasets (e.g., TinyGSM, MetaMath) or scaling up model parameters significantly (e.g., 70B+ models), which is computationally expensive and limits generalization to distribution shifts (robustness).

The authors propose that instead of scaling data or model size, performance can be improved through adaptive inference-time architecture that dynamically routes problems, diversifies reasoning styles, and aggregates results based on uncertainty.

2. Methodology: The AMR Framework

The Adaptive Multi-Expert Reasoning (AMR) framework consists of four core components designed to operate during inference without requiring additional synthetic training data.

A. Difficulty-Aware Router

The router analyzes the input problem text to predict two key metrics:

1. Difficulty: Estimated based on the problem text (at inference time) or reference solution length (during training/evaluation).
2. Uncertainty: Calculated using a hybrid entropy–margin formulation:
   $$U(x) = \frac{1}{2}H(p(x)) + \frac{1}{2}(1 - 2|p_{hard}(x) - 0.5|)$$
   Where $H(p(x))$ is Shannon entropy and $p_{hard}$ is the probability of the problem being "hard."

Based on the uncertainty score $U(x)$, the system adapts its generation strategy:

• Low Uncertainty ($U < 0.35$): Deterministic generation (single path).
• Medium Uncertainty ($0.35 \le U < 0.55$): One candidate per expert with low temperature.
• High Uncertainty ($U \ge 0.55$): Two candidates per expert with varied temperatures (0.0 and 0.15) to maximize diversity.

B. Multi-Expert Reasoning

The system employs three specialized LoRA-adapted experts, each trained with distinct prompts to generate different reasoning paradigms:

1. Algebraic: Focuses on equation-based reasoning.
2. Intuitive: Relies on mental math and natural language.
3. Step-by-step: Provides detailed, structured derivations.

The pipeline includes two refinement passes:

• Correction Pass: The step-by-step expert attempts to fix errors in the best initial candidates.
• Finalizer Pass: Generates a concise, high-quality final solution.

C. Neural Verifier

A binary classifier (based on DeBERTa-v3) evaluates the correctness of candidate solutions. Trained on problem-solution pairs, it assigns a confidence score (0–1) representing the likelihood of correctness. This allows for the filtering of candidates that do not provide a clear answer indicator.

D. Clustering-Based Aggregation

Instead of simple majority voting, AMR uses a sophisticated scoring and clustering mechanism:

1. Candidate Scoring: Each candidate receives a weighted score:
   $$Score = 0.50 \cdot s_{verifier} + 0.18 \cdot c_{completion} + 0.16 \cdot q_{quality} + 0.16 \cdot b_{source}$$
   • $s_{verifier}$: Verifier confidence.
   • $c_{completion}$: Bonus for structured formatting (e.g., "####" or "final answer").
   • $q_{quality}$: Coherence based on answer length.
   • $b_{source}$: Bonus for candidates from correction/finalizer passes.
2. Clustering: Candidates are grouped by their extracted numerical answers.
3. Cluster Selection: The final cluster score combines the best candidate's score, the cluster's average score, expert consensus, and cluster size. The best candidate from the highest-scoring cluster is selected.

3. Key Contributions

1. Difficulty-Sensitive Routing: A mechanism that dynamically controls the breadth of generation (number of candidates and diversity) based on predicted problem difficulty and uncertainty.
2. Multi-Expert Framework: Integration of stylistically specialized LoRA experts with correction and finalization passes to enhance robustness and output quality.
3. Uncertainty-Guided Aggregation: A novel clustering approach that balances verifier confidence, answer quality, and expert consensus to select the final answer.
4. Data Efficiency: Demonstrates that high performance can be achieved using only the original training data (GSM8K), avoiding the need for massive synthetic data augmentation.

4. Results

The model was evaluated on the GSM8K test split (1,319 examples) using a 7B parameter base model (Qwen2.5-Math).

• Overall Accuracy: 75.28%.
• Performance Breakdown:
  • Easy problems (predicted): 82.6%
  • Hard problems (predicted): 64.1%
• Comparative Analysis:
  • AMR outperformed the majority of comparable 7B models trained on extensive synthetic data (e.g., MetaMath-7B at 66.7%, WizardMath-7B at 54.9%, ToRA-Code-7B at 72.6%).
  • It approaches the performance of larger 13B models trained on synthetic data.
  • It significantly outperforms base models without math tuning (e.g., LLaMA-2-7B at 14.6%, Mistral-7B at 52.2%).

5. Significance and Conclusion

The paper establishes that inference-time architecture is a critical, underutilized lever for improving LLM reasoning.

• Robustness: By relying on diversity and uncertainty modeling rather than memorization of synthetic patterns, AMR offers a different type of robustness against distribution shifts (though GSM-PLUS evaluation is proposed as future work).
• Efficiency: The results challenge the prevailing "bigger is better" or "more data is better" paradigm, showing that a 7B model with smart routing and aggregation can outperform larger models trained on millions of synthetic examples.
• Future Directions: The authors plan to extend the framework to broader benchmarks (MATH, SVAMP), implement dynamic expert selection, and rigorously test robustness on perturbed datasets like GSM-PLUS.

In summary, AMR proves that adaptive decision-making during inference—specifically routing based on difficulty and aggregating based on uncertainty—is a highly effective strategy for mathematical reasoning, offering a cost-effective alternative to massive data scaling.