Bayesian Uncertainty Quantification for Ranked Choice Voting Polls

Technical Summary: Bayesian Uncertainty Quantification for Ranked Choice Voting Polls

Problem Statement
Ranked Choice Voting (RCV) presents unique methodological challenges for pollsters that traditional frequentist uncertainty quantification (UQ) tools cannot adequately address. In RCV, the election winner is determined by a path-dependent elimination process where candidates are successively removed and votes reallocated until a majority is secured. Unlike plurality (First-Past-The-Post) elections, where the winner is a function of a single population parameter (the vote share), the RCV winner is a union of disjoint pathways determined by the full structure of voter preference rankings.

Consequently, standard metrics like the "margin of error" (MoE) fail to capture the complexity of RCV uncertainty. A MoE typically quantifies uncertainty around a specific elimination margin, but it does not directly quantify the probability of a candidate winning. Furthermore, in RCV, a candidate who is the true winner in the population may be the most frequent winner in small-sample polls due to sampling variability crossing decision boundaries in the high-dimensional parameter space. Conversely, standard frequentist confidence sets for the vector of elimination margins often become so wide that they overlap with multiple potential winner regions, failing to identify a unique winner even at large sample sizes.

Methodology
The authors propose a Bayesian framework that shifts the inferential target from estimating specific elimination margins to estimating the posterior probability of victory for each candidate. The methodology proceeds as follows:

1. Modeling Ballot Orderings: The problem is cast in terms of "ballot orderings" ($P$), the set of all unique ways a voter can rank candidates. The population distribution of these orderings is denoted by $\pi$. The RCV winner is a deterministic function of $\pi$.
2. Conjugate Inference: The authors model the observed poll responses as independent draws from a multinomial distribution over the set of ballot orderings. By utilizing the conjugate relationship between the Multinomial likelihood and the Dirichlet prior, they derive a posterior distribution for $\pi$.
   • If $n_j$ is the count of observed ballots of type $j$, and the prior is $\pi \sim \text{Dir}(\alpha)$, the posterior is $\pi | \text{data} \sim \text{Dir}(\alpha + n)$.
   • This allows for analytical tractability without requiring complex computational methods for the posterior itself.
3. Ballot Pruning: To address the "curse of dimensionality" where the number of possible ballot orderings grows super-exponentially with the number of candidates, the authors introduce a "ballot pruning" lemma. This lemma posits that if the set of candidates remaining in the final rounds of an election is known, the relative rankings of eliminated candidates do not affect the final winner. The authors recommend a bootstrap procedure to identify a stable set of top candidates (e.g., the top 3 or 4) and prune the ballot space to only include rankings relevant to these candidates. This reduces the dimensionality of the Dirichlet distribution while preserving the integrity of the winner determination.
4. Simulation Pipeline:
   • Prune the ballot space based on bootstrap stability.
   • Compute the posterior Dirichlet distribution using the observed (or weighted) counts.
   • Repeatedly sample from this posterior to generate simulated electorates.
   • Run the RCV tabulation algorithm on each simulated electorate.
   • The proportion of simulations in which a candidate wins constitutes their estimated probability of victory.

Key Results and Applications
The authors validate their approach using two high-profile RCV elections: the 2021 New York City Democratic mayoral primary and the 2022 Alaska U.S. House special election.

• Alaska 2022 Special: The authors demonstrate that in unbiased simulations of small sample sizes, the true winner (Mary Peltola) is often not the most probable winner in the polls. In fact, Nick Begich (who was eliminated first in the actual election but would have won a head-to-head against Peltola) appears as the most frequent winner in small samples. This highlights the "fragility" of RCV point estimates. Standard frequentist confidence ellipsoids for the elimination margins were found to overlap all three winner regions in nearly 97% of simulations, rendering them useless for winner identification.
• NYC 2021 Primary: The authors re-analyzed a final poll by Data for Progress. While the poll's point estimates suggested a tight race between Eric Adams and Kathryn Garcia (with Maya Wiley trailing), the Bayesian analysis provided a nuanced uncertainty measure. The posterior win probabilities were estimated at 56.4% for Adams, 42.8% for Garcia, and 0.8% for Wiley. This correctly identified the race as a two-person contest with Adams as a modest favorite, consistent with the actual 0.8% margin of victory.
• Weighted Polls: The framework was extended to handle survey weights by treating weighted counts as pseudo-counts in the Dirichlet update, allowing the method to be applied to real-world polling data that employs demographic weighting.

Significance and Claims
The paper claims that the proposed Bayesian framework offers a superior alternative to standard frequentist practices for RCV polls by directly targeting the quantity of interest: the probability of winning.

• Direct Interpretation: Unlike margins of error, which require readers to infer the likelihood of a winner from a range of margins, the posterior win probabilities provide a direct, continuous measure of uncertainty regarding the election outcome.
• Handling Path Dependency: The method naturally "bakes in" the path-dependent nature of RCV eliminations. It accounts for the fact that uncertainty in early elimination orders may or may not propagate to the final winner, a nuance lost in standard margin-based reporting.
• Avoiding False Certainty: The authors argue that point estimation in RCV is fraught because incorrect winners can be favored in a plurality of random samples. The Bayesian approach avoids forcing binary conclusions (e.g., "statistically significant lead") and instead communicates the relative likelihood of victory, which is more appropriate for the complex geometry of RCV decision boundaries.
• Feasibility: The approach is computationally tractable due to the conjugacy of the Dirichlet-Multinomial pair and the dimensionality reduction provided by ballot pruning.

The authors conclude that while their method shows promise in heavily contested elections, it serves as a foundational step for more robust uncertainty quantification in RCV, moving the field away from inadequate margin-of-error reporting toward probabilistic statements about election outcomes.