CUPID in the Model Zoo: Online Matchmaking for Selecting Your Dream LLM

Technical Summary: CUPID in the Model Zoo

Problem Statement
As Large Language Models (LLMs) proliferate, users face a critical challenge in selecting the most appropriate model for specific tasks from a rapidly expanding pool. This selection is complicated by two factors: (1) LLMs possess distinct but often opaque latent properties (e.g., reasoning quality, verbosity, latency, cost-efficiency) that are not fully observable a priori; and (2) users often lack the vocabulary or awareness to explicitly articulate their preferences, which are frequently application-specific and latent rather than generic.

Existing solutions primarily focus on query-level routing, where individual prompts are dynamically assigned to models based on fixed utility definitions (e.g., accuracy vs. latency). However, routing does not address the problem of identifying a single LLM for sustained deployment where user preferences are unknown, latent, and must be inferred through limited interaction under strict cost and time budgets. Current methods often assume known objectives or rely on population-level aggregates, failing to capture individual user needs.

Methodology: The CUPID Framework
The authors propose CUPID (Cost-aware User-centric Preference Identification and Discovery), an interaction-efficient active learning framework designed to match users to their optimal LLM. The framework operates under three core desiderata: application-centricity (learning individual preferences), user-centricity (using lightweight feedback), and budget-awareness (respecting cost and round limits).

The core algorithm is HEART-UCB (Human-Elicited Adaptive Resource-aware Tradeoffs), a novel Upper Confidence Bound strategy derived for latent dueling bandits. The methodology proceeds as follows:

1. Latent Preference Modeling: User preferences are modeled as a latent function $f: \mathcal{Z} \times \mathcal{A} \to \mathbb{R}$, where $\mathcal{A}$ is the set of LLMs and $\mathcal{Z}$ represents a set of unknown, latent user objectives (e.g., "cheap," "verbose"). A Gaussian Process (GP) prior is placed over this utility function to represent uncertainty over the entire model pool.
2. Belief Updating: The system maintains a belief vector $b_t$ over the latent objectives. Upon observing a pairwise preference (e.g., Model A is preferred over Model B), the belief over objectives is updated via Bayes' rule, and the GP posterior over utilities is updated using a probit likelihood.
3. HEART-UCB Formulation: To select the next pair of models for comparison, the algorithm computes a belief-aware UCB score for each model. This score integrates:
   • Expected Utility: The posterior mean of the latent utility.
   • Exploration Bonus: The posterior standard deviation (uncertainty).
   • Language-Driven Bias: An optional term derived from natural language feedback (e.g., "I need a cheaper model"). An auxiliary LLM interprets this feedback to generate indicator functions that bias the selection toward models aligning with the stated direction.
   • Resource Penalty: A dynamic penalty term that increases if the algorithm spends budget faster than a target rate, effectively penalizing expensive models when the budget is tight.
   • Cooldown: A term to prevent repeated selection of the same model when candidates have comparable utility.
4. Interaction Loop: CUPID iteratively selects a pair of LLMs, presents their responses to the user, collects pairwise feedback (and optional natural language cues), and updates the belief and GP posterior until the budget is exhausted or convergence is reached.

Key Contributions

1. Framework Formulation: CUPID is formulated as a general framework for matching users to LLMs under latent, hard-to-articulate preferences while respecting explicit cost and time budgets.
2. HEART-UCB Algorithm: The authors derive HEART-UCB, a theoretically grounded online algorithm that extends standard UCB to latent dueling bandits. It uniquely integrates language feedback as a biasing signal and incorporates resource constraints directly into the utility calculation.
3. Empirical Validation: The paper demonstrates the effectiveness of CUPID through extensive experiments on text and image generation tasks, comparing it against baselines including $\epsilon$-greedy dueling bandits, RUCB, and LMArena-style sampling.

Experimental Results

• Automated Validation: In simulations using a pool of 25 OpenAI models with varying objective sets, CUPID achieved 100% convergence across all experimental setups, significantly outperforming baselines like RUCB and Latent Dueling Bandits, which frequently failed to converge. CUPID also demonstrated superior cost-efficiency, achieving high convergence rates with lower total API costs.
• Robustness: CUPID showed robustness to noisy language feedback (adversarial, static, or gibberish) and objective shifts (training on general benchmarks like MMLU-Pro while evaluating on specific reasoning tasks like AIME). It consistently outperformed baselines in both score and cost under these conditions.
• Human Studies: In A/B-blinded user studies involving text and image generation, participants rated CUPID's final matched models as having comparable or better quality than those selected by LMArena, while CUPID achieved significantly higher budget compliance ratings. The advantage of CUPID was most pronounced in settings with latent, open-ended user objectives (H3 and H4), suggesting it effectively infers hidden preferences where explicit specifications are lacking.
• Scalability: The algorithmic overhead of CUPID (GP updates and UCB scoring) scales linearly with the model pool size but remains negligible (<1 second for 125 models) compared to LLM generation latency.

Significance and Claims
The paper claims to be the first attempt to (i) formulate latent dueling bandits using UCB and (ii) integrate language feedback into online interaction for user-LLM matchmaking. The authors position CUPID as a solution for the "complementary regime" of user-level model selection, distinct from per-query routing.

The significance of the work lies in its ability to enable efficient discovery of LLMs that satisfy individual user needs with minimal interaction overhead and cost. By treating user objectives as latent variables and updating beliefs through lightweight pairwise comparisons and optional natural language cues, CUPID reduces the reliance on users having precise technical vocabulary or pre-defined specifications. The authors note that while the current GP formulation does not automatically analyze response content, future work could incorporate content-aware embeddings to further accelerate convergence. The framework is presented as a step toward more informed, efficient, and cost-effective deployment decisions for both individuals and organizations.