FUSE: Ensembling Verifiers with Zero Labeled Data

1. Problem Statement

Large Language Models (LLMs) increasingly rely on test-time scaling techniques, such as Best-of-N (BoN) sampling, where multiple responses are generated for a single query, and a verifier (or reward model) selects the best one.

• The Challenge: In practice, ground-truth labels for correctness are expensive and time-consuming to acquire. Consequently, systems rely on imperfect verifiers (external LLMs or reward models).
• The Limitation of Current Methods:
  • Simple Aggregation: Naive methods like majority voting or averaging scores often fail because they treat unreliable verifiers equally to strong ones.
  • Semi-Supervised Methods: Existing advanced ensembling techniques (e.g., WEAVER) typically require a small set of labeled data (ground truth) to learn verifier weights.
  • Theoretical Gaps: Classical unsupervised ensemble learning methods (e.g., Jaffe et al., 2015) rely on strong conditional independence assumptions (specifically Joint Conditional Independence, or JCI) between verifiers. In LLM settings, verifiers are often correlated (e.g., trained on similar data), violating these assumptions and causing performance degradation.

Goal: Develop a method to ensemble multiple imperfect verifiers to select the correct LLM response without any access to ground-truth labels, while handling conditional dependencies between verifiers.

2. Methodology: FUSE (Fully Unsupervised Score Ensembling)

FUSE is a three-step pipeline that adapts spectral ensemble learning to the LLM verification setting. It operates on a matrix $V$ of scores from $m$ verifiers for $N$ generated responses.

Step 1: Adaptive Score Transformation (Minimizing TCI Violations)

Standard spectral methods assume Triplet Conditional Independence (TCI): given the true label, any three verifiers' scores are independent. However, LLM verifiers often violate this.

• Mechanism: FUSE introduces an empirical measure of TCI violation derived from the covariance tensors of verifier scores.
• Action: It searches for a monotone transformation $g_\tau$ (specifically, binarizing scores using optimal thresholds $\tau$) that minimizes the empirical TCI violation statistic.
• Result: The raw scores are transformed into a binary matrix $\tilde{V}$ where the TCI assumption holds sufficiently well for the next step.

Step 2: Method-of-Moments (MoM) Estimation

Using the transformed matrix $\tilde{V}$, FUSE applies the spectral algorithm from Jaffe et al. (2015).

• Input: Second-order (covariance) and third-order (cumulant) tensors of the verifier outputs.
• Output: Consistent estimates of sensitivity ($\psi$) and specificity ($\eta$) for each verifier, conditioned on the query.
• Key Insight: By first transforming the data to satisfy TCI, the MoM estimator can recover verifier qualities even when the raw data is dependent.

Step 3: Pseudo-Label Construction and Ensemble Optimization

Instead of assuming Joint Conditional Independence (JCI) to derive a closed-form ensemble (which is often false), FUSE uses the estimated sensitivities/specificities to construct pseudo-labels.

• Posterior Estimation: It estimates the posterior probability of correctness $P(y=1 | \text{verifier scores})$ by averaging estimates derived from all possible triplets of verifiers (using the TCI assumption).
• Optimization: These pseudo-labels serve as "soft" ground truth to train a parametric ensemble model (e.g., Logistic Regression). The model parameters $\theta$ are optimized to maximize the agreement between the ensemble's predictions and the estimated pseudo-labels.
• Selection: The final response is selected as the one with the highest predicted correctness score from the optimized ensemble.

3. Key Contributions

1. Zero-Shot Ensembling: FUSE is the first method to effectively ensemble diverse LLM verifiers for response selection without requiring any ground-truth labels, outperforming simple baselines and competing with semi-supervised methods.
2. Relaxing Independence Assumptions: The paper identifies that standard unsupervised methods fail in LLM settings due to verifier correlations. FUSE introduces a novel adaptive transformation step that empirically minimizes TCI violations, making spectral estimation robust to real-world dependencies.
3. Pseudo-Labeling Framework: Instead of relying on the untestable JCI assumption for ensemble construction, FUSE uses the estimated verifier qualities to generate pseudo-labels, allowing for the optimization of flexible, data-adaptive ensemble rules (like logistic regression).
4. Broad Applicability: The method works with both discrete (binary) and continuous (real-valued) verifier scores and supports both query-conditional (per-question) and batched processing modes.

4. Experimental Results

The authors evaluated FUSE on diverse benchmarks ranging from standard academic tests to frontier, unsaturated challenges.

• Datasets:

  • Saad-Falcon et al. (2025b) Data: GPQA, MATH500, MMLU, MMLU-Pro (using Llama 3.1/3.3 generators).
  • Humanity's Last Exam (HLE): A 649-question subset of extremely difficult, unsaturated problems.
  • IMO Shortlist: Modified International Math Olympiad questions.

• Performance Highlights:

  • vs. Semi-Supervised Baselines: In 70B and 8B model settings, FUSE achieved parity or slight improvements over WEAVER (a leading semi-supervised method that uses 5% labeled data) across most benchmarks.
  • vs. Unsupervised Baselines: FUSE decisively outperformed Majority Vote and Naive Ensemble (simple averaging).
    • Example: On MMLU-Pro (70B), FUSE achieved 91.4% accuracy vs. 87.0% for Naive Ensemble and 74.4% for Majority Vote.
  • Extreme Difficulty (HLE): On the Humanity's Last Exam, where naive ensemble performed worse than random guessing (Pass@1), FUSE achieved 54.3% accuracy, outperforming the Oracle Best Verifier (53.5%) and Logistic Regression (53.4%).
  • Homogeneous Verifiers (IMO): Even when verifiers were highly similar (violating diversity assumptions), FUSE matched the performance of a Naive Ensemble (which acts as an oracle in this specific case) while other methods lagged.

• Ablation Studies:

  • FUSE showed superior robustness in "mixed" settings (heterogeneous tasks) compared to semi-supervised methods that suffer from distribution shift when labels are scarce or domain-specific.

5. Significance and Impact

• Reducing Reliance on Human Labels: FUSE demonstrates that high-quality verification can be achieved without the prohibitive cost of human labeling, a critical bottleneck for scaling LLM training and deployment.
• Theoretical Advancement: It bridges the gap between theoretical unsupervised ensemble learning (which assumes independence) and the practical reality of correlated LLM verifiers, providing a robust statistical framework for the former.
• Future Applications: The authors suggest FUSE can be applied to:
  • Unsupervised Model Ranking: Scoring models on unsolved problems without ground truth.
  • Benchmark Auditing: Detecting label errors in existing datasets by identifying inconsistencies between estimated and key correctness.
  • Data Filtering: Selecting high-quality synthetic data for training RLHF or synthetic datasets.

In summary, FUSE represents a significant step toward fully unsupervised, scalable verification for large language models, proving that sophisticated statistical ensembling can overcome the lack of ground truth and the presence of verifier correlations.