Technical Summary: Learning What Evaluators Value

Problem Statement

In numerous domains—including academic peer review, medical diagnosis, admissions, and product ratings—evaluators map assessments of multiple criteria (e.g., soundness, novelty, cleanliness) to an overall score or recommendation. The fundamental challenge addressed in this paper is learning the preference function that governs this mapping.

Current approaches typically rely on strong parametric assumptions, most notably linearity (or extensions like generalized additive models with interactions). The authors argue that these assumptions are frequently violated in practice. Citing Jones et al. (2025), they note that researchers often fail to verify linearity assumptions, and Keswani et al. (2025) demonstrate that human moral preferences often involve thresholds and non-linearities that linear models cannot capture. The paper posits that assuming an incorrect functional form leads to three critical failures:

1. Inability to learn preferences: The estimated function may fail to approximate the true preference function, leading to high estimation error.
2. Ranking errors: The estimated function may produce rankings of items (e.g., papers, applicants) that disagree significantly with the true ranking.
3. Misunderstanding criteria importance: Regression coefficients in linear models may incorrectly infer the relative weight of criteria, especially when sample distributions are imbalanced.

Methodology

Theoretical Framework

The authors adopt a non-parametric approach, making only a minimal assumption: the preference function is isotonic (coordinate-wise non-decreasing). This means that if a criterion score increases while others remain constant, the overall score should not decrease. This class, denoted as $\mathcal{F}_{ISO}$, is expressive enough to capture interactions, thresholds, and non-linear transformations.

The paper first establishes negative results for common parametric models ($\mathcal{F}_{GENINT}$, which includes linear, generalized additive, and interaction models):

• Estimation Error: There exist isotonic preference functions where the best possible estimator within $\mathcal{F}_{GENINT}$ incurs a constant-factor error (up to random guessing) compared to the true function.
• Ranking Error: For certain isotonic functions, the Kendall tau distance between the ranking produced by a generalized additive model and the true ranking is bounded away from zero (constant fraction of pairs ranked incorrectly).
• Coefficient Bias: In imbalanced datasets, ordinary least squares (OLS) can infer that a criterion is more important than another even when the reverse is true or when the underlying process is symmetric.

Proposed Algorithm

To address these issues without sacrificing performance when linearity does hold, the authors propose a Regularized Isotonic Regression algorithm (Algorithm 1). The core idea is to balance two objectives via cross-validation:

1. Fit to Data: Minimizing the squared error on the training set.
2. Model Mismatch Penalty: Minimizing the distance between the learned isotonic function and the best-fitting linear function.

Algorithm Steps:

1. Regularization: The algorithm solves a regularized least squares problem over the set of isotonic functions ($\mathcal{F}_{ISO}$) and linear functions ($\mathcal{F}_{LIN}$). It introduces a regularization parameter $\lambda$ that interpolates between pure isotonic regression ($\lambda=0$) and linear regression ($\lambda=\infty$).
   $$\min_{\hat{f} \in \mathcal{F}_{ISO}} \min_{g_{LIN} \in \mathcal{F}_{LIN}} \frac{1}{|S'|} \sum_{j \in S'} (\hat{f}(x^{(j)}) - y^{(j)})^2 + \frac{\lambda}{|X_{S'}|} \sum_{x \in X_{S'}} (\hat{f}(x) - g_{LIN}(x))^2$$
2. Post-Processing: The output function is truncated to the valid range $[0, 1]$ and interpolated for unseen inputs to ensure monotonicity.
3. Cross-Validation: The optimal $\lambda$ is selected from a set $\Lambda$ using a hold-out validation set to minimize validation risk.

Theoretical Guarantees

The authors prove that their algorithm satisfies two competing goals:

1. Robustness: For any monotonic preference function, the estimation error of their algorithm vanishes as the sample size $N \to \infty$ (converging at a rate of $O(N^{-1/d})$).
2. Efficiency: If the true preference function is linear, the algorithm performs nearly as well as non-negative least squares (linear regression), with the risk upper-bounded by the risk of linear regression up to a logarithmic factor dependent on the number of regularization parameters.

Key Results

Synthetic Simulations

The algorithm was tested on synthetic data generated from three utility models: Linear, Leontief (perfect complements), and Cobb-Douglas.

• Linear Preferences: Both the proposed algorithm and linear regression achieved comparable estimation errors.
• Non-Linear Preferences: For Leontief and Cobb-Douglas utilities, linear regression failed to reduce error beyond a certain sample size (showing bias), while the proposed algorithm continued to learn the function, albeit at a slower rate. At $N=1000$, the error of linear regression was over 13 times larger than the proposed algorithm for Leontief preferences.

Real-World Human Preferences (Tripadvisor)

The algorithm was applied to hotel ratings from Tripadvisor, predicting overall scores from six criteria (value, rooms, location, cleanliness, service, sleep quality).

• Prediction Error: The proposed algorithm showed a modest but statistically significant improvement in prediction error (approx. 8–11% reduction) compared to linear regression.
• Reducible Error: More significantly, the algorithm reduced the reducible error (a proxy for estimation error, calculated by subtracting irreducible noise) by 50% to 69% compared to linear regression. This indicates the algorithm learns the underlying community preference function much more accurately.
• Insights: The learned functions revealed non-linearities, such as diminishing returns (concavity) for high scores and significant impacts of low scores, which linear models (assuming constant marginal changes) could not capture.

LLM vs. Human Preferences (ICLR Peer Review)

Using a dataset of human and LLM (GPT-4o, Llama 3.1 70b) reviews for ICLR 2024 papers:

• Consistency: GPT-4o exhibited significantly lower irreducible error (higher internal consistency) than Llama and human evaluators.
• Alignment: GPT-4o's preference function was significantly closer to human preferences than Llama's.
• Shape Differences: The learned preference functions showed that GPT-4o's mapping from criteria to overall scores was non-linear (concave or convex depending on other scores), whereas human preferences appeared closer to linear in the observed plots.

Significance and Claims

The paper claims to provide a reliable, robust approach to learning evaluator preferences that avoids the pitfalls of model misspecification.

• Theoretical Contribution: It establishes that common parametric assumptions can lead to worst-case errors equivalent to random guessing for ranking and estimation, and that linear models can fundamentally misinterpret criteria importance.
• Practical Contribution: The proposed algorithm offers a "best of both worlds" solution: it is provably robust to complex, non-linear preferences while maintaining performance comparable to linear regression when the linear assumption holds.
• Application: The authors demonstrate that this method can reveal nuanced insights into human and LLM decision-making (e.g., diminishing returns, threshold effects) that are invisible to standard linear models. They suggest this is particularly valuable for downstream tasks like ranking or aligning AI systems with human values, where understanding the structure of the preference function is more critical than mere prediction accuracy.

The authors remain modest regarding prediction accuracy gains in noisy real-world datasets, noting that the primary benefit lies in the faithful estimation of the underlying preference function rather than just predicting scores. They conclude that their method enables researchers to learn "what evaluators care about" without missing morally or practically relevant non-linear components of their beliefs.