📈 economics

Scaling the Queue: Reinforcement Learning for Equitable Call Classification Capacity in NYC Municipal Complaint Systems

This paper proposes an equity-centered reinforcement learning framework for New York City's municipal complaint systems that optimizes call classification capacity across six Department of Buildings domains by acting as an intelligent router to maximize throughput, minimize errors, and actively reduce historical racial and income-based service disparities.

Original authors: Irene Aldridge, Ellie Bae, Siddhesh Darak, Nicholas Donat, Akhil Fernando-Bell, Bella Ge, Nicholas Goguen-Compagnoni, Ishita Gupta, Ali Hasan, Pierce Hoenigman, Imran Isa-Dutse, Jiwon Jeong, Tishya Kh

Published 2026-05-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Irene Aldridge, Ellie Bae, Siddhesh Darak, Nicholas Donat, Akhil Fernando-Bell, Bella Ge, Nicholas Goguen-Compagnoni, Ishita Gupta, Ali Hasan, Pierce Hoenigman, Imran Isa-Dutse, Jiwon Jeong, Tishya Khanna, Neha Konduru, Yixuan Liu, Kai Maeda, Nolan McKenna, Karl Muller, Farzaan Naeem, Rishabh Patel, Zachary Sheldon, Ammar Syed, Nathan Tai, Michael Twersky, Haoying Wang, Zening Wang, Zexun Yao, Nadav Yochman

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine New York City's 311 system as a giant, chaotic waiting room. Every day, thousands of people call to report problems: a broken boiler, a shaky scaffold, a lack of heat, or a leaking pipe. The city has a limited number of inspectors (the "doctors" in this waiting room) to fix these issues.

The problem is that the current system for deciding who gets seen first is broken. It relies on a simple rule: "More calls mean more urgency." But this rule has a hidden flaw. Wealthy neighborhoods with better internet, more English speakers, and higher trust in the government tend to call 311 more often and faster. Poorer neighborhoods, where people might not have the same resources or trust, call less often, even if their buildings are just as dangerous.

Because the city's current computer system just counts the calls, it ends up sending inspectors to rich neighborhoods (where there are many calls) and ignoring poor neighborhoods (where there are fewer calls, but the problems are just as real). It's like a triage nurse who only treats the patients who are shouting the loudest, while the ones sitting quietly in pain get forgotten.

The Solution: A "Fairness Coach"

The authors of this paper built a new kind of computer brain using Reinforcement Learning (RL). Think of this not as a robot replacing the human staff, but as a super-smart "Fairness Coach" sitting next to the intake workers.

Instead of just telling the system to "fix as many problems as possible," the researchers taught this coach a new goal: "Fix problems fairly, no matter where they are."

Here is how they did it, using simple analogies:

1. The "Ghost Call" Correction
The biggest trick in this paper is how they handle the "missing calls." The researchers realized that just because a poor neighborhood hasn't called 311 doesn't mean the buildings are safe. It just means the residents haven't been able to call yet.

The Analogy: Imagine a teacher grading a class. If one student raises their hand 10 times and another raises it once, the teacher might think the first student is more engaged. But what if the second student is too shy to speak up?
The Fix: The computer uses a special math formula (based on neighborhood data like income and education) to estimate the "Ghost Calls." It asks, "If everyone in this neighborhood called as easily as the rich neighborhood, how many calls would we actually have?" It then uses this estimated number to decide who needs help, rather than just the raw number of calls received.

2. The "Fairness Score"
The computer learns by playing a game. Every time it sends an inspector to a building, it gets points.

Old System: You get points for sending inspectors to anyone who called.
New System: You get points for sending inspectors to anyone who needs help, but you get extra points if you help a neighborhood that usually gets ignored. If the system starts ignoring poor neighborhoods, it loses points. This forces the computer to balance speed with fairness.

3. The Trade-off (The "Price" of Fairness)
The researchers asked: "Does being fair make the whole system slower?"

The Result: Yes, but only a tiny bit. By adding the fairness goal, the system became about 4% to 7% slower at processing calls overall. However, this small slowdown meant that low-income neighborhoods got the help they needed much faster, closing the gap between rich and poor areas. The authors argue this small cost is worth it to stop the system from discriminating against the poor.

4. The "Black Box" Problem
Sometimes, computers make decisions we don't understand. To fix this, the team used a tool called SHAP (which acts like a magnifying glass). They looked at why the computer made its choices.

The Discovery: They found that the computer was paying attention to things like "neighborhood zip code" and "language spoken." While this helped it find the "Ghost Calls," it also meant the computer was indirectly using race and income to make decisions. The team says this is okay only if we keep watching it closely to make sure it's helping people, not hurting them.

The Bottom Line

The paper concludes that you cannot just build a faster computer to fix city problems; you have to build a fairer one.

Without the fairness goal: The computer gets faster but keeps ignoring the poor.
With the fairness goal: The computer gets slightly slower overall, but it stops ignoring the people who need help the most.

The authors emphasize that this isn't just a technical fix; it's a moral choice. By programming the computer to value fairness, the city can ensure that a broken boiler in a poor neighborhood gets fixed just as quickly as one in a rich neighborhood, even if the poor neighborhood called less often.

In short: The paper shows that with the right "rules of the game," we can teach computers to be fair, ensuring that the city's help reaches everyone, not just the people who are best at asking for it.

Technical Summary: Scaling the Queue – Reinforcement Learning for Equitable Call Classification Capacity in NYC Municipal Complaint Systems

1. Problem Statement

Municipal 311 call centers and complaint intake systems face a structural mismatch between incoming complaint volume and the capacity available to classify, triage, and route them. Current intake workflows rely on reactive, rule-based heuristics that fail to scale with demand. This bottleneck results in systematic misprioritization, where complaints from low-income and minority neighborhoods are disproportionately assigned to low-priority queues, batched inappropriately, or left unclassified.

The core issue is not merely random error but measurement injustice. Complaint volume is a biased signal of underlying housing risk because reporting rates are positively correlated with income, education, and digital literacy. Consequently, classifiers trained naively on historical 311 data inherit and amplify the "participation gap," where under-served communities generate fewer records despite potentially higher incident rates. This leads to a feedback loop where under-reported areas receive fewer inspections, generating even fewer outcome records for future training.

The paper addresses three research questions:

Equity: Can Reinforcement Learning (RL) policies distribute correct action assignments equitably across demographic groups?
Capacity: Can RL agents achieve higher throughput and lower misclassification rates than existing heuristics?
Trade-offs: What is the quantitative cost of equity, and how does the Pareto frontier between speed, cost, and equitable coverage shift with reward configuration?

2. Methodology

2.1 Formal Framework: Multi-Objective MDP

The authors formalize six New York City Department of Buildings (DOB) operational domains (Boiler Safety, Crane/Derrick, Heat/Hot Water, Housing Triage, Scaffold Safety, and Natural Area Districts) as discrete-time Markov Decision Processes (MDPs).

State ( $S$ ): Encodes the complaint queue, neighborhood statistics (census-tract level), and capacity indicators. Features include recurrence flags, backlog, and neighborhood risk scores.
Action ( $A$ ): Classification categories such as Escalate, Batch, Defer, Inspect Now, or Ignore.
Reward ( $R$ ): A multi-objective function designed to maximize throughput, minimize misclassification costs, and actively narrow historical equity gaps.

2.2 Equity-Centered Reward Design

A critical innovation is the treatment of equitable classification coverage as a first-class reward objective rather than a side constraint. The reward function at step $t$ is defined as:
$r_t = \alpha_1 r_{speed} - \alpha_2 r_{cost} + \alpha_3 r_{equity} + \alpha_4 r_{retention}$

To address the "participation gap," the authors introduce an upstream reporting-rate correction before normalizing escalation throughput. Instead of using raw complaint counts ( $N_g$ ) as the denominator for equity calculations, they estimate the latent incident count ( $\hat{N}_g$ ) using:

Duplicate-Report Estimator: Following Liu et al. [12], the rate of duplicate reports for the same incident is used to estimate the reporting propensity ( $\rho$ ) of a neighborhood.
Bayesian Spatial Model: When duplicate data is sparse, ACS covariates (income, education, language access) proxy for reporting propensity.
Correction: The estimated latent incident count $\hat{N}_g = C_g / \hat{\rho}_g$ replaces raw counts in the equity term. This ensures the agent is incentivized to escalate complaints from low-reporting (often low-income) tracts proportionally to their estimated need, not their filing volume.

2.3 Algorithms and Training

The study evaluates several RL algorithms against baselines (current DOB heuristics, random policies, and rule-based expert systems):

DQN (Deep Q-Network): Used with experience replay and target networks; includes CQL (Conservative Q-Learning) penalties in specific domains.
REINFORCE with Learned Baseline: A policy-gradient method optimized for sequential resource allocation.
Tabular TD Q-Learning: Used for smaller state spaces.
Behavioral Cloning (BC): Used as a baseline to estimate historical policy performance.

Training utilizes strict chronological splits (e.g., 2020–2024 for training, 2025 for testing) to prevent data leakage and simulate real-world deployment.

2.4 Interpretability and Auditing

The authors employ SHAP (SHapley Additive exPlanations) attribution to audit the learned policies. This identifies whether features acting as proxies for demographic attributes (e.g., neighborhood income, racial composition) are driving decisions. The audit is disaggregated by census-tract income quintile and racial composition to detect disparate impacts.

3. Key Results

3.1 Classification Throughput and Accuracy

Heat and Housing Domains: REINFORCE demonstrated a 4x improvement in cumulative reward compared to HDQN and random policies. It closed approximately 20–27% more violations (101 vs. 80–84) over a 365-day horizon.
Boiler Safety: DQN outperformed REINFORCE in total reward ($10.3M vs. $9.0M) and recall (98.7% vs. 94.5%), though both exhibited low precision (~22%) due to the high cost asymmetry of missing a safety hazard (26:1 cost ratio).
Elevator and SNAD Domains: RL agents (DQN and REINFORCE) consistently outperformed the "Balanced" heuristic and other baselines in terms of reward and actionable classification rates.

3.2 Multi-Objective Trade-offs (Pareto Frontier)

The "Price of Equity": The study finds that adding an equity objective ( $\alpha_3 > 0$ ) results in a modest throughput penalty of 4–7%. This aligns with Liu and Garg's [13] theoretical bounds, suggesting that efficiency and equity are largely complementary rather than adversarial in municipal settings.
Policy Dominance: RL policies with equity objectives dominate the status quo NYC inspection scheduling on both efficiency and equity dimensions simultaneously.

3.3 Equity Analysis

Disparity Reduction: The current DOB heuristic exhibits a statistically significant gradient where low-income neighborhoods (Q1) receive fewer correct escalations per complaint than high-income neighborhoods (Q5).
Equity-Centric RL: Agents trained with the corrected equity reward term ( $\alpha_3 > 0$ ) approach parity across income quintiles. In contrast, agents trained without the equity term narrow but do not close the gap.
Feature Importance: SHAP analysis revealed that neighborhood-level features (e.g., Community Board 12 in Manhattan) are strong predictors, confirming that geographic variables in historical data encode socioeconomic status.

4. Significance and Claims

The paper claims that equity is not an inevitable byproduct of efficiency; it must be explicitly inscribed into the reward function.

Necessity of Explicit Equity: Agents optimized solely for throughput narrow historical disparities but fail to eliminate them. Only agents with explicit coverage parity objectives achieve near-parity while outperforming heuristics on defect detection.
Modest Cost: The trade-off for equity is quantifiable and small (4–7% throughput loss), challenging the notion that fairness requires significant operational sacrifice.
Data Quality Limits: The authors emphasize that classification quality is bounded by data quality. Without correcting for the "participation gap" (under-reporting in low-income areas), RL systems risk degrading equity over successive training rounds due to feedback loops.
Normative Policymaking: Configuring reward functions for public-sector RL is an act of normative policymaking. The paper argues that the choice of where to operate on the Pareto frontier (balancing speed, cost, and equity) is a political decision that belongs to the agency and the communities it serves, not just the algorithm.

5. Deployment Recommendations

The authors propose a "Human-in-the-Loop" deployment strategy with specific safeguards:

Ongoing Audits: Monthly demographic audits with pre-specified thresholds for manual review.
Reporting Correction: Applying the duplicate-report estimator to correct equity denominators before every retraining cycle.
Feedback Loop Mitigation: Using exploration bonuses and counterfactual escalations to prevent the model from reinforcing historical under-service.
Transparency: Publicly documenting reward configurations as statements of normative policy priority.

The paper concludes that while RL offers a lever to scale classification capacity, realizing the promise of equitable service delivery requires correcting the structural biases embedded in civic data collection before those biases shape algorithmic decisions.