ModeX: Evaluator-Free Best-of-N Selection for Open-Ended Generation

1. Problem Statement

Large Language Models (LLMs) are inherently stochastic, meaning that generating multiple responses to the same prompt yields diverse outputs. While sampling multiple paths (Best-of-N) and aggregating them (e.g., via Self-Consistency) has proven effective for closed-ended tasks (like multiple-choice questions), it remains a fundamental challenge for open-ended generation (e.g., summarization, code, creative writing).

Existing Best-of-N selection methods face two primary limitations:

1. Reliance on External Evaluators: Most methods use external reward models or LLM-as-a-Judge to score candidates. This introduces significant computational overhead, latency, and dependency on auxiliary models.
2. Inapplicability to Open-Ended Text: Traditional voting schemes rely on exact string matching. In open-ended tasks, high-quality answers may be semantically identical but lexically distinct (paraphrased), causing exact-match voting to fail.

The core question addressed is: Can we select a single high-quality output from multiple generation paths without external evaluators or significant computational overhead?

2. Methodology: ModeX

The authors propose ModeX (Mode Extraction), a framework that generalizes the principle of majority voting to open-ended text by identifying the "modal" output—the generation that represents the dominant semantic consensus among samples.

Core Algorithm

ModeX operates in three steps, treating the set of generated texts as a graph:

1. Adjacency Matrix Construction:

   • Nodes represent generated responses.
   • Edges represent pairwise similarity. Instead of exact matches, ModeX uses a weighted sum of Jaccard similarities over unigrams, bigrams, and trigrams ($s_1, s_2, s_3$).
   • This creates a soft generalization of voting, allowing semantically similar but lexically different texts to be connected.

2. Recursive Spectral Clustering:

   • The goal is to isolate the dominant semantic cluster (the "mode") from outliers (hallucinations or errors).
   • ModeX constructs the Graph Laplacian ($L = D - A$) and computes the Fiedler vector (the second smallest eigenvector).
   • The Fiedler vector provides a continuous relaxation of the minimum cut problem, effectively splitting the graph into two subgraphs.
   • Recursive Partitioning: The algorithm recursively splits the graph until the conductance ratio ($\phi$) of a cut exceeds a threshold $\tau$ (set to 0.8). Conductance measures the quality of the cut; a high value indicates the subgraphs are well-connected internally and poorly connected to each other.
   • The recursion stops when further splitting does not yield a meaningful separation, leaving a final cluster of mutually consistent outputs.

3. Centroid Selection:

   • From the final dominant cluster, ModeX selects the centroid.
   • The centroid is defined as the node with the maximum weighted degree (sum of edge weights to all other nodes in the cluster).
   • This node represents the output most similar to the consensus of the cluster, serving as the approximation of the "modal" generation.

Extension: ModeX-Lite

To improve efficiency, the authors introduce ModeX-Lite.

• Pruning Strategy: Instead of generating full sequences and then clustering, ModeX-Lite generates paths in parallel and applies the clustering/pruning step at fixed intervals (e.g., every $T=100$ tokens).
• Non-Recursive Clustering: During generation, it performs spectral clustering only once per interval (without recursion) to prune non-representative trajectories early.
• Result: This retains the robustness of multi-path aggregation while significantly reducing the computational cost of generating full "bad" trajectories.

3. Key Contributions

1. Evaluator-Free Framework: ModeX is the first Best-of-N selection framework for open-ended generation that does not require external reward models, LLM judges, or exact string matching.
2. Theoretical Generalization: The paper provides a theoretical justification (Theorems 1 & 2) showing that spectral clustering isolates high-density semantic manifolds, and degree centrality approximates Kernel Density Estimation (KDE) to find the mode. This formally generalizes "majority voting" to continuous semantic spaces.
3. Efficiency Optimization: ModeX-Lite demonstrates that high-quality candidates can be identified and pruned early in the generation process, balancing robustness with low latency.
4. Comprehensive Evaluation: The method is validated across three diverse open-ended tasks: Text Summarization (CNN/DailyMail), Code Generation (HumanEval), and Mathematical Reasoning (Math-500).

4. Experimental Results

The authors evaluated ModeX and ModeX-Lite against several baselines, including Single Path, Self-Refine, LLM Judge, Perplexity, and Self-Certainty.

• Performance Gains:
  • Code Generation: On HumanEval with Qwen-7B, ModeX (N=16) improved Pass@1 from 69.89% (Single Path) to 78.66%, outperforming the LLM Judge baseline (70.12%) and approaching the Gold Standard Best-of-N (82.00%).
  • Math Reasoning: ModeX consistently outperformed baselines on Math-500, showing that structure-aware selection is more effective than simply generating more candidates.
  • Summarization: ModeX achieved state-of-the-art results among evaluator-free approaches on CNN/DailyMail (Rouge-L scores consistently higher than Single Path and Self-Refine).
• Comparison to Baselines:
  • Vs. Self-Refine: Self-Refine (iterative correction) often degraded performance or failed to surpass Single Path, highlighting that without a selection mechanism, self-correction can propagate errors.
  • Vs. LLM Judge: ModeX outperformed LLM-as-a-Judge, proving that internal consensus is a more reliable signal than external ranking.
  • Scalability: Increasing the number of generation paths ($N$) yielded consistent gains for ModeX, whereas baselines like LLM Judge showed diminishing returns.
• Efficiency:
  • Latency: ModeX-Lite (N=16) achieved a 3.5x speedup over Self-Refine and was faster than LLM Judge because it avoids a second inference pass.
  • Complexity: The selection step is $O(N^2)$, which is negligible compared to the $O(NL)$ generation cost (where $L$ is sequence length).

5. Significance and Implications

• Inference-Time Scaling: The paper argues that improvements in LLM performance come not just from scaling model parameters or sampling more, but from structured aggregation. Leveraging the intrinsic distributional structure of model outputs (consensus) is a powerful, low-cost scaling strategy.
• Practical Deployment: By removing the dependency on external reward models (which are expensive and hard to train for open-ended tasks), ModeX makes robust Best-of-N selection accessible for real-world applications.
• Robustness: The method effectively filters out hallucinations and verbose errors, which tend to appear as sparse outliers in the semantic graph, while preserving the "modal" correct reasoning.

6. Limitations

• Lexical Similarity: ModeX relies on Jaccard similarity (n-grams). It may struggle with high-quality paraphrases that have very different surface forms but identical meaning. The authors suggest embedding-based similarity as a potential future improvement.
• Systematic Bias: If the underlying LLM has a "mode collapse" toward a specific hallucination or bias, the spectral clustering will reinforce this error, as it selects the most frequent consensus. This is a limitation of the model's distribution rather than the selection algorithm itself.

In conclusion, ModeX offers a principled, computationally efficient, and evaluator-free approach to selecting high-quality outputs from LLMs, bridging the gap between the success of voting in closed-ended tasks and the complexities of open-ended generation.