A Unified Perturbation Framework for Analyzing Leaderboard Stability and Manipulation

Technical Summary: A Unified Perturbation Framework for Analyzing Leaderboard Stability and Manipulation

Problem Statement

Evaluation leaderboards, such as LMArena and Chatbot Arena, are central to benchmarking Large Language Models (LLMs) by aggregating pairwise human preferences into rankings using the Bradley–Terry (BT) framework. While these platforms drive model adoption and industry recognition, the robustness of their rankings against structured data modifications remains poorly understood. Existing literature has identified isolated vulnerabilities—such as sensitivity to vote injection, data leakage, or the removal of specific comparisons—but lacks a unified formalism to analyze how different types of perturbations propagate through the ranking estimator to affect diverse objectives (e.g., top-k membership, global consistency, and statistical uncertainty). This paper addresses the question of how small, structured changes to comparison datasets impact leaderboard conclusions across multiple dimensions.

Methodology

The authors propose a unified influence-based perturbation framework for Bradley–Terry leaderboards. The core of the method involves using influence functions to approximate the local effect of modifying match weights without requiring full re-fitting of the model for every candidate action.

1. Influence Propagation:
The framework treats dataset modifications as structured interventions on the weighted BT estimator. It utilizes influence functions to approximate the change in the latent skill vector $\hat{\theta}$ resulting from a perturbation to a match weight $w_n$:
$$\frac{\partial \hat{\theta}(w)}{\partial w_n} \bigg|_{w=1} = -H^{-1} g_n$$
where $H$ is the Hessian of the loss and $g_n$ is the gradient for match $n$. These parameter sensitivities are propagated to scalar leaderboard objectives $f(\hat{\theta}, w)$. To account for the finite nature of the perturbations (rather than infinitesimal changes), the authors employ a one-step Newton (1sN) refinement to partially correct for curvature changes in the perturbed objective.

2. Action Space:
The framework defines three fundamental match-level perturbations and one player-level intervention:

• Drop: Removing an existing match ($w_n: 1 \to 0$).
• Add: Introducing a new match ($w_{new}: 0 \to 1$). The authors consider three candidate spaces with increasing levels of control: all_pairs (outcome fixed by skill), all_outcomes_weighted (outcomes weighted by BT probability), and all_outcomes (outcome fully controlled).
• Flip: Reversing the outcome of an existing match ($y_n \to 1-y_n$), modeled as dropping the original and adding the reversed match.
• Player Removal: Dropping all matches incident to a specific player, analyzed via grouped Newton refinement.

3. Evaluation Objectives:
The framework evaluates perturbations against three distinct classes of objectives:

• Top-k Membership (Gap Objective): Measures the skill difference ($\theta_i - \theta_j$) across top-k boundaries to determine if a model enters or leaves the top-k set.
• Global Ranking Consistency (Kendall's $\tau$): Uses a differentiable tanh-based surrogate to measure the correlation between the perturbed ranking and the original reference ranking.
• Statistical Uncertainty (Confidence Intervals): Uses a local information approximation (inverse Fisher information) to estimate the variance of skill estimates. Objectives include reducing the uncertainty of a specific player ($f_{CI-player}$) or the global trace uncertainty ($f_{CI-trace}$).

Key Contributions

1. Unified Framework: The paper introduces a single formalism that decouples the perturbation mechanism (Drop, Add, Flip) from the audited objective, allowing for the simultaneous study of robustness failures and targeted interventions.
2. Systematic Non-Robustness: Through experiments on seven datasets (including Chatbot Arena, MT-Bench, and sports rankings like NBA and ATP), the authors demonstrate that modern leaderboards are non-robust. Sub-1% targeted perturbations are sufficient to change the top-ranked model, degrade global ranking consistency, and alter confidence intervals.
3. Efficient Targeted Manipulation: The same influence scores enable efficient targeted interventions. The framework can promote or demote specific models with fewer actions than previous vote-rigging baselines (e.g., omni-rigging) and reduce target-model uncertainty more effectively than active-sampling baselines (e.g., Arena Active).
4. Player Removal Analysis: The study quantifies the "leaderboard illusion" by showing that removing influential players (e.g., highly connected models) can induce broad reordering of the remaining models, shifting ranks and altering top-10 memberships.
5. Normalized Robustness Scores: The authors aggregate influence-guided failures into normalized dataset-level robustness scores ($R_{Top-1}$, $R_{CI}$, $R_{\tau}$), providing a compact audit profile across different views of leaderboard stability.

Results

• Robustness: On major LLM leaderboards (e.g., Arena 55k), fewer than 1% of targeted actions (specifically Flip or Drop) can change the Top-1 model. For instance, only 3 Flip actions or 5 Drop actions were needed to alter the Top-1 on Arena 55k.
• Objective Sensitivity:
  • Flip is generally the most action-efficient perturbation for changing rankings and degrading Kendall's $\tau$.
  • Drop is more conservative but still highly effective.
  • Add variants are less effective for changing rankings but are the most effective for reducing uncertainty (constructive intervention).
• CI-Aware Robustness: While accounting for confidence intervals increases the budget required to change a ranking (by an average of 13$\times$ compared to point estimates), leaderboards remain vulnerable. Targeted drops can still alter top-k membership under strict CI-aware criteria.
• Dataset Variance: Expert-curated datasets (e.g., MT-Bench) exhibit higher robustness than crowdsourced arena-style datasets. Sparse graphs (e.g., ATP Top-10) show the lowest global ranking stability.
• Player Removal: Removing a single highly influential player can shift up to 28 players in rank and change up to 6 top-10 memberships, with "degree" (number of matches) being the strongest predictor of a player's influence.

Significance and Claims

The paper claims to provide a practical and helpful tool for auditing leaderboard stability. By bridging robustness analysis and adversarial manipulation, the framework reveals fundamental limitations in current leaderboard designs. The authors emphasize that the same techniques used to audit fragility could theoretically be used to manipulate rankings, highlighting a dual-use tension.

The significance of this work lies in its ability to:

1. Motivate more robust evaluation protocols by quantifying the fragility of current systems.
2. Provide a defensive mechanism, where low robustness scores or highly influential comparisons can trigger conservative reporting, targeted data collection, or delayed updates.
3. Offer a unified perspective that extends beyond single-perturbation studies, covering match addition, removal, and flipping across multiple ranking objectives.

The authors remain modest regarding the scope, acknowledging that influence estimates are local approximations and that their uncertainty objectives rely on scalable proxies that may not capture all sources of annotation noise or judge bias. They suggest future work should focus on higher-order attribution methods and outlier-robust estimators.