🤖 machine learning

Regime-Calibrated Demand Priors for Ride-Hailing Fleet Dispatch and Repositioning

This paper introduces a training-free, regime-calibrated framework that segments historical ride-hailing data into demand regimes and matches current conditions using a similarity ensemble to generate demand priors, which drive LP-based repositioning and batch dispatch to reduce mean rider wait times by 31.1% across diverse scenarios while generalizing effectively to new cities without retraining.

Original authors: Indar Kumar, Akanksha Tiwari

Published 2026-04-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Indar Kumar, Akanksha Tiwari

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the captain of a massive fleet of taxis in a busy city like New York. Your job is to make sure that whenever someone hails a ride, a driver is nearby to pick them up quickly.

The problem? The city is unpredictable.

On a rainy Tuesday morning, everyone rushes to work.
On a sunny Saturday night, everyone is heading to bars.
On New Year's Eve, the whole city goes crazy.

Most taxi companies try to guess where the next ride will come from using complex computer models that learn over time. But these models are like black boxes: they are expensive to train, hard to understand, and sometimes they get confused when something unusual happens (like a sudden snowstorm or a surprise concert).

This paper proposes a smarter, simpler way to solve this. Think of it as "Learning from the City's Memory."

The Core Idea: "What Happened Last Time?"

Instead of trying to predict the future with a crystal ball, the authors say: "Let's look at the past."

They realized that while every day is unique, the patterns of the city repeat. A busy Wednesday morning in January feels very similar to a busy Wednesday morning in June. The authors call these repeating patterns "Regimes."

Here is how their system works, broken down into three simple steps:

1. Building the "City Memory Bank" (The Regime Library)

Imagine the city's history is a giant library. The authors took 5.2 million taxi trips and sorted them into 4-hour "chapters" (Regimes).

Chapter A: "Winter Morning Rush."
Chapter B: "Summer Friday Night Party."
Chapter C: "New Year's Eve Chaos."

Each chapter has a detailed map of where people were going, how fast they were moving, and how many cars were needed.

2. The "Matchmaker" (Similarity Ensemble)

Now, imagine it's 8:00 AM on a Tuesday in January. The system looks at the current traffic and asks: "Which chapter in our library does this look like?"

It doesn't just guess. It uses a six-point checklist to find the best match:

Shape: Does the curve of requests look similar?
Distance: Are the pickup spots in the same neighborhoods?
Variability: Is the traffic as chaotic or as smooth as the past?
Events: Is there a special event happening (like a parade)?
Time: Is it the same time of day and day of the week?
History: How close in time was this past event?

It finds the top 5 "chapters" from history that match the current moment and blends them together. This creates a Calibrated Prior—a super-accurate guess of what will happen next, based on what actually happened in similar situations before.

3. The "Smart Dispatcher" (LP Repositioning)

Once the system knows what the future looks like, it doesn't just wait for orders. It gets proactive.

Imagine the drivers are like water in a bucket. If you know a drought is coming in the North and a flood in the South, you move the water before the drought starts.

The system uses a math formula (Linear Programming) to tell idle drivers: "Don't wait here. Drive 10 blocks North right now. We know a rush is coming there in 15 minutes."
When a real request comes in, it uses a classic matching algorithm (Hungarian algorithm) to pair the rider with the closest driver instantly.

Why Is This Better?

The authors tested this on real New York City data and compared it to standard methods. The results were like magic:

Faster Rides: On average, riders waited 31% less time.
Fairness: The people who usually wait the longest (the "tail" of the distribution) saw their wait times drop by 37%. It made the service fairer for everyone.
No Training Needed: Unlike AI models that need to "study" for months, this system works immediately. It just looks at the library and matches patterns.
It Travels: They tested it in Chicago using the New York library, and it still worked! The city patterns are similar enough that the "memory" transferred perfectly.

The "Aha!" Moment

The most surprising finding was that simpler is often better.
The system had a complex "six-metric" checklist. But when they tested it, they found that just looking at the shape of the demand (how many people want rides) was actually the most important part. The fancy "event" and "calendar" checks sometimes added noise.

However, they kept the full system because it's robust—it handles weird edge cases (like New Year's Eve) that simple math might miss.

The Bottom Line

This paper teaches us that sometimes you don't need a super-complex AI to solve a problem. You just need a good librarian who knows how to find the right historical story to match the current situation.

By matching the "vibe" of today with the "vibe" of similar days in the past, the system can move drivers to the right place before the rush even starts, saving thousands of hours of waiting time for riders everywhere.

1. Problem Statement

Ride-hailing platforms face significant challenges in demand uncertainty. Request patterns are non-stationary, shifting drastically based on time-of-day, day-of-week, season, weather, and special events (e.g., New Year's Eve).

Current Limitations: Most operational algorithms either assume stationary demand (failing during volatility) or rely on black-box machine learning forecasters (e.g., Deep RL, LSTMs) that require costly retraining, lack explainability, and struggle with regime shifts.
Core Challenge: How to anticipate demand patterns and optimize fleet repositioning without heavy training overhead, while ensuring robustness across diverse scenarios and cities.

2. Methodology

The authors propose a Regime-Calibrated approach that avoids training a predictive model. Instead, it retrieves and adapts historical demand patterns. The system operates in four stages:

A. Regime Library Construction

Data Segmentation: Historical trip data (NYC TLC, 5.2M trips) is segmented into 4-hour blocks (regimes) indexed by date and time.
Features: Each block stores demand time-series (5-min bins), summary statistics (mean, skewness, autocorrelation), spatial Origin-Destination (OD) coordinates, and event annotations (surge detection).
Library Size: 373 blocks covering winter/summer, weekdays/weekends, and holidays.

B. Similarity Ensemble (Regime Matching)

When a new operating period begins, the system matches the current demand pattern to the most similar historical regimes using a six-metric similarity ensemble:

Kolmogorov–Smirnov (KS): Distributional distance.
Wasserstein-1 (W1): Earth mover's distance for distribution shape.
Feature Distance: Euclidean distance in summary feature space.
Variance Ratio: Comparing volatility.
Event Pattern: Similarity in surge intensity and duration.
Temporal Proximity: Bonuses for matching month, day-type, and hour.

Mechanism: A weighted sum of these metrics identifies the top- $k$ (default $k=5$ ) historical analogues. An adaptive gate handles cases where one side has zero events to avoid misleading comparisons.

C. Calibrated Demand Prior

The top- $k$ matched regimes are combined to form a calibrated demand prior:

Temporal Profile: A weighted average of demand rates ( $\hat{\lambda}(t)$ ) based on similarity scores.
Spatial Profile: Raw trip coordinates from matched regimes are pooled to preserve spatial correlation structures.
Volume Matching: The prior is scaled to match the current block's observed request count to prevent systematic over/under-prediction.

D. Optimization & Dispatch

The calibrated prior drives two optimization layers:

LP-Based Anticipatory Repositioning: A min-cost transportation Linear Programming (LP) problem is solved every 3 minutes. It redistributes idle drivers from surplus zones to deficit zones (predicted hotspots) to minimize travel time while maximizing demand coverage.
Batch Dispatch: Pending requests are matched to idle drivers every 60 seconds using the Hungarian algorithm on a cost matrix of estimated pickup times.

3. Key Contributions

Regime-Calibrated Demand Prior: A novel, training-free method that uses a six-metric similarity ensemble to retrieve historical analogues, creating a demand prior with spatial and temporal fidelity.
LP-Based Repositioning: A provably optimal min-cost transportation LP that utilizes the calibrated prior to proactively move idle drivers, with theoretical bounds on unserved demand relative to forecast accuracy.
Cross-City Generalization & Robustness: The method generalizes to Chicago without retraining and remains robust across fleet size scalings (0.5x to 2x).
Formal Validation: Comprehensive statistical testing (Friedman, Nemenyi, Cohen's $d$ ) and ablation studies proving the multiplicative value of calibration and LP repositioning.

4. Experimental Results

Evaluated on 5.2 million NYC trips across 8 diverse scenarios (winter/summer, weekdays/weekends, holidays, rush hours) with 5 random seeds each.

Performance Gains:
- Mean Wait Time: Reduced by 31.1% (95% CI: [26.5%, 36.6%]) compared to a baseline of replay + batch matching.
- Tail Performance: P95 wait times dropped by 37.6%, significantly improving service for the most disadvantaged riders.
- Fairness: The Gini coefficient of wait times improved from 0.441 to 0.409 (7.3% relative improvement), indicating more equitable service.
Component Decomposition:
- Calibration alone: Reduced wait by 16.9%.
- LP Repositioning alone (on top of calibration): Reduced wait by an additional 15.5%.
- The two components act multiplicatively.
Cross-City Transfer: Applied to Chicago data using the NYC library, achieving a 23.3% reduction in mean wait time without any local retraining.
Statistical Significance: Friedman test $\chi^2 = 80.0$ ( $p = 4.25 \times 10^{-18}$ ); Cohen's $d$ effect sizes ranged from 7.5 to 29.9 (extremely large).
Efficiency: The system runs in < 5 seconds per 4-hour simulation on a standard CPU. The overhead of calibration and LP is negligible (~0.25s).

5. Significance and Implications

Explainability & Determinism: Unlike Deep Reinforcement Learning (RL) approaches which are black boxes and require massive compute, this method is deterministic, explainable, and requires zero training.
Handling Volatility: The method excels in high-volatility scenarios (e.g., New Year's Eve, Friday nights) where traditional stationary assumptions fail, achieving up to 45.6% wait reduction in these specific periods.
Theoretical Soundness: The paper provides formal proofs for the LP's optimality and the error bounds of the similarity-weighted calibration, demonstrating that similarity weighting is theoretically superior to uniform averaging.
Practical Deployment: The approach is computationally lightweight and robust to routing fidelity (OSRM vs. Haversine), making it highly suitable for real-time production deployment in ride-hailing platforms.

Conclusion: The paper demonstrates that a "retrieval-based" approach, grounded in regime matching and linear optimization, can outperform complex learning-based methods in ride-hailing dispatch, offering a robust, fair, and highly efficient solution for fleet management.