🤖 AI

A Comparative Simulation Study of the Fairness and Accuracy of Predictive Policing Systems in Baltimore City

This paper presents a comprehensive simulation study in Baltimore City revealing that while predictive policing systems can be more accurate and fair than traditional hot spots policing in the short term, both approaches suffer from similar biases and feedback loops, with predictive systems potentially amplifying inequities over the long run.

Original authors: Samin Semsar, Kiran Laxmikant Prabhu, Gabriella Waters, James Foulds

Published 2026-02-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Samin Semsar, Kiran Laxmikant Prabhu, Gabriella Waters, James Foulds

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine Baltimore as a giant, complex garden. Some parts of the garden are full of weeds (crime), while others are clean. The city's police force is the gardening crew, and their job is to decide where to send the weed-whackers.

For a long time, the crew used a simple method called "Hot Spots Policing." It's like looking at the garden yesterday and saying, "Wow, there were a lot of weeds here yesterday, so I'll send the crew there again today." It's a reactive approach based on recent history.

Then, technology gave them a new tool: "Predictive Policing." This is like a fancy weather forecast for weeds. It uses complex math to look at patterns over a longer time and predicts exactly where weeds will grow tomorrow, hoping to catch them before they spread.

The big question everyone was asking was: Is this new, high-tech tool fairer than the old way? Does it treat all neighborhoods equally, or does it unfairly target specific groups of people?

This paper is like a giant, 300-day video game simulation where the researchers played out the role of the police in Baltimore to see what happens. They didn't just guess; they built a digital twin of the city and ran the simulation 20 times to see how the "weed-whackers" (police) were distributed over time.

Here is what they found, explained simply:

1. The "Short-Term" vs. "Long-Term" Trap

The researchers tested three different "gardening strategies":

Short-Term KDE: Looking only at the weeds from the last month.
Long-Term KDE: Looking at the weeds from the last year.
PredPol (Predictive Policing): The fancy algorithm that learns and predicts.

The Surprise: The fancy new tool (PredPol) was actually the best at finding the weeds (accuracy) and was generally the fairest at the start. It spread the police officers more evenly than the old "look at last month" method.

2. The "Feedback Loop" (The Vicious Cycle)

Here is the tricky part. The paper describes a feedback loop like a snowball rolling down a hill.

If you send more police to a neighborhood, they find more crimes (because they are looking harder).
The computer sees "More crimes found!" and sends even more police there next time.
This creates a cycle where the police keep piling up in the same spots, regardless of whether that's actually where the most crime is happening.

The Catch: While the fancy tool (PredPol) started out fairer, it rolled down the hill faster. It amplified bias (unfairness) more quickly than the older methods. If you run the simulation for a very long time, the fancy tool might end up being the least fair because it gets stuck in that cycle so fast.

3. The "Baltimore Twist" (It's Not Always About Race)

Most people assume these systems always unfairly target Black neighborhoods. The researchers found something surprising in Baltimore:

When they used data on all types of crimes, the simulation showed the police were actually being sent more to White neighborhoods (like Downtown) over time.
However, when they looked only at Aggravated Assaults, the police were sent more to Black neighborhoods.

The Analogy: Think of the algorithm like a GPS. If you tell the GPS "Find me the busiest roads," it might take you to a wealthy suburb. If you tell it "Find me the most dangerous intersections," it might take you to a different part of town. The tool isn't inherently "evil"; it just follows the map you give it. In Baltimore, the map changed the destination.

4. The "Fairness vs. Accuracy" Trade-off

The paper found that being "accurate" (finding the most crimes) doesn't automatically mean being "fair."

Sometimes, a system can be very good at finding crimes but very unfair in how it treats neighborhoods.
Sometimes, a system can be fair and accurate at the same time.
The Lesson: You can't just look at one number. You have to check both the "crime-catching score" and the "fairness score" separately.

5. The Main Takeaway

The researchers are saying: "Don't just buy the shiny new tool and hope for the best."

Before a city like Baltimore (or any city) starts using these predictive systems in real life, they should run their own simulations first. Just like you wouldn't build a bridge without testing the blueprints, you shouldn't deploy a police algorithm without testing how it behaves in your specific city's unique history and demographics.

In a nutshell: The new predictive tools are powerful and can be fairer than the old ways, but they have a "speed dial" for unfairness. If you don't watch them closely, they can accidentally create a cycle where they keep punishing the same neighborhoods over and over again. The solution isn't to throw the tools away, but to test them carefully in a "practice garden" before planting them in the real world.

Technical Summary: A Comparative Simulation Study of the Fairness and Accuracy of Predictive Policing Systems in Baltimore City

Problem Statement
Predictive policing systems, designed to enhance crime prediction accuracy over traditional "hot spots" policing, have faced scrutiny regarding racial bias and unfairness. While prior research (e.g., Lum and Isaac [41]) has demonstrated that these systems can amplify bias through feedback loops when trained on historically biased data, there is a lack of comprehensive comparative studies evaluating predictive policing against traditional hot spots methods. Crucially, the relative fairness and accuracy of these approaches have not been established, particularly in the context of Baltimore, Maryland. Baltimore presents a unique case study due to its history of racial segregation, the "New Jim Crow" dynamics of mass incarceration, and its early adoption of predictive policing technologies around 2018. The specific dynamics of how police presence affects Black versus White neighborhoods—where presence in Black communities often induces fear and instability rather than perceived safety—necessitates a localized evaluation to determine if predictive models reinforce existing disparities or offer a fairer alternative.

Methodology
The authors developed an agent-based simulation framework to evaluate the long-term fairness and accuracy of three policing algorithms over a 300-day period using Baltimore City crime data from 2019–2022.

Algorithms Compared:
1. PredPol: A representative predictive policing model based on the Epidemic-Type Aftershock Sequence (ETAS) framework, which treats crime as a self-exciting point process.
2. Short-term KDE: A Kernel Density Estimation model representing traditional hot spots policing, utilizing only the most recent month of crime history.
3. Long-term KDE: A KDE model utilizing one year of crime history to ensure a fair data-volume comparison with PredPol.
Simulation Mechanics:
- Data: The study utilized 207,447 crime records from Open Baltimore.
- Detection and Reporting: The simulation introduced stochasticity to model real-world policing limitations. A "Noisy-OR" function determined if a crime was detected based on the number of officers present ( $k$ ) and a detection probability ( $p=0.5$ ). A separate coin flip with a weighted probability (0.4, derived from Bureau of Justice Statistics data) determined if a detected crime was reported.
- Feedback Loop: Police allocation for day $t$ was determined by the model trained on crimes detected and reported up to day $t-1$ . This created a dynamic feedback loop where police presence influenced detection, which in turn influenced future training data.
- Scenarios: The simulation ran across 8 distinct settings varying the number of officers (40 vs. 400), crime reporting probability (0 vs. 0.4), and crime type (all crimes vs. aggravated assault only).
Metrics:
- Fairness: Measured via Racial Fairness Gaps (absolute difference in average police share and police-to-crime ratio between Black and White neighborhoods) and Neighborhood-Level Fairness (Gini coefficients of police distribution and police-to-crime ratios).
- Accuracy: Measured as Coverage Accuracy (the proportion of total crimes detected).

Key Contributions

Contextualized Simulation: The study provides the first systematic simulation analysis of predictive policing specifically within Baltimore, addressing a gap in literature that has largely focused on cities like Oakland or Los Angeles.
Comparative Framework: It establishes a logical basis for comparing predictive policing (PredPol) against traditional hot spots policing (KDE variants) regarding both accuracy and fairness over time.
Bias Dynamics Analysis: The work investigates how bias evolves over time, distinguishing between short-term performance and long-term tendencies driven by feedback loops.
Data-Driven vs. Model-Driven Bias: The study demonstrates that bias outcomes are not solely determined by the algorithm but are significantly influenced by the specific crime data fed into the model (e.g., aggravated assault vs. all crimes).

Results

Accuracy: PredPol was generally the most accurate algorithm, achieving the highest coverage accuracy in 87.5% of scenarios. Short-term KDE was consistently the least accurate.
Short-term Fairness: In the short term, PredPol exhibited higher racial and neighborhood-level fairness compared to both KDE variants. Long-term KDE improved upon short-term KDE in accuracy and neighborhood fairness but performed worse on racial fairness metrics.
Long-term Bias Amplification: While PredPol started as the fairest model, it demonstrated the fastest rate of bias amplification in most scenarios. The feedback loop caused PredPol's fairness to degrade more rapidly than the KDE models.
Counter-Intuitive Bias Direction: Contrary to the prevailing assumption that predictive policing biases against Black neighborhoods, the simulation using "all crime records" in Baltimore showed a trend toward over-policing White neighborhoods (e.g., Downtown, Mount Vernon, Canton) in the long run. However, when using only "aggravated assault" records, the bias trended toward Black neighborhoods (e.g., Sandtown-Winchester, Blair-Edison).
Correlation: A correlation was observed between neighborhood-level fairness and accuracy (higher fairness gaps often accompanied lower accuracy), but no such correlation existed between racial fairness and accuracy.
Volatility: Short-term KDE showed the most volatile changes in neighborhood-level fairness, while PredPol showed the most consistent (though rapidly worsening) bias amplification trends.

Significance and Claims
The paper claims that the situation regarding bias in predictive policing is more complex than previously assumed. The authors assert that:

Traditional methods are not inherently fairer: Hot spots policing (KDE) exhibits similar bias issues to predictive policing, challenging the notion that simpler models are a safe alternative.
Context matters: The direction of bias (toward Black or White neighborhoods) is contingent on the specific crime data used and the local demographic landscape, rather than being an intrinsic property of the algorithm alone.
Trade-offs exist: While predictive policing may offer superior accuracy and initial fairness, its susceptibility to rapid feedback-loop amplification suggests potential for worse long-term behavior.
Methodological Necessity: The study advocates for the use of simulation methodologies as a prerequisite for policy decisions. Authorities and community stakeholders should utilize such localized evaluations to navigate the complexities of fairness before deploying predictive policing tools, as real-world implementation without pre-evaluation risks reinforcing inequities.

The authors conclude that while predictive policing was, on average, fairer and more accurate than hot spots policing in their 300-day simulation, the speed of bias amplification warrants caution. They recommend that future deployments in Baltimore and similar contexts be preceded by localized simulation studies to identify potential long-term tendencies and inequities.