💻 computer science

Accurate Pedestrian Tracking in Urban Canyons: A Multi-Modal Fusion Approach

This paper presents a particle filter-based multi-modal fusion system combining GNSS, RoNIN inertial data, and spatial map priors to significantly improve pedestrian localization accuracy and sidewalk assignment in urban canyons, particularly for blind or low-vision users where GNSS signals are degraded.

Original authors: Shahar Dubiner, Peng Ren, Roberto Manduchi

Published 2026-02-02

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Shahar Dubiner, Peng Ren, Roberto Manduchi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: Getting Lost in the "Concrete Canyon"

Imagine you are walking through a city filled with skyscrapers. These tall buildings create what the paper calls an "urban canyon."

If you try to use your phone's GPS (GNSS) here, it's like trying to listen to a radio station while standing inside a deep, narrow well. The tall buildings block the signal from the satellites, or the signal bounces off the glass walls (like an echo) before reaching your phone. This makes the GPS location very shaky and inaccurate.

This is a big problem for everyone, but especially for people who are blind or have low vision. They rely entirely on the phone to tell them, "You are on the left side of the street; wait for the light to cross." If the phone thinks you are on the right side of the street when you are actually on the left, the guidance is dangerous.

The Solution: A Team of Three

The researchers built a system that acts like a detective team to figure out where you are, even when the GPS is confused. They combined three different tools:

The GPS (The Shaky Compass): It gives a general idea of where you are, but it's often wrong in the city.
The Inertial Sensor (The Step Counter): Your phone has sensors that feel your movement. The paper uses a smart AI called RoNIN that watches how you walk. It knows, "You took 10 steps forward," regardless of whether you are holding the phone in your hand or it's bouncing in your pocket. It's great at tracking movement, but if you walk for 10 minutes without a GPS check, it starts to drift off course (like a compass that slowly spins).
The Map (The Rules of the Road): This is the secret sauce. The system has a digital map that knows where the impenetrable buildings are and where the streets are. It knows you can't walk through a brick wall, and you probably aren't walking in the middle of a busy intersection unless you are jaywalking.

How It Works: The "Party Guest" Game

The paper uses a technique called Particle Filtering. Imagine you are trying to guess where a friend is in a crowded room, but you can't see them.

The Particles: The system creates 500 invisible "guesses" (particles) scattered around the map. Each guess is a possible location for you.
The Movement: As you walk, the system moves all 500 guesses forward based on your step count (from the RoNIN AI).
The Rules (Spatial Priors):
- If a guess lands inside a building, the system says, "Impossible!" and kills that guess (weight = 0).
- If a guess lands on a sidewalk, the system says, "Very likely!" (weight = 1).
- If a guess lands in the middle of the street (jaywalking), the system says, "Maybe, but unlikely," and gives it a lower score (weight = 0.4).
The GPS Check: When the GPS sends a signal, the system checks the 500 guesses. If a guess is close to the GPS signal, it gets a boost. If it's far away, it gets penalized.
The Result: The system keeps the "good" guesses and throws away the "bad" ones. The final answer is the average of all the surviving guesses.

The Analogy: Think of it like a game of "Hot and Cold." The GPS says "You are near here," but the map says "You can't be in that building." The system uses the map to filter out the impossible answers, leaving only the most likely spot where you could actually be standing.

The Experiment: Walking in San Francisco

The researchers tested this in downtown San Francisco, a city famous for its tall buildings. They walked six different routes, including:

Busy streets with skyscrapers.
A covered transit center (like a giant bus station roof) where GPS signals are almost non-existent.

They compared three methods:

GPS Only: Just the shaky compass.
Inertial Only: Just the step counter + the map rules (no GPS).
The Fusion: The step counter + the map rules + the GPS.

The Results: Who Won?

The paper measured success in three ways:

Sidewalk Assignment: Did the system know which side of the street you were on? (This is the most important metric).
Total Distance Error: How many meters off were you?
Street Direction Error: Were you wrong about being across the street or along the street?

The Findings:

GPS Alone: Failed miserably. It only identified the correct sidewalk 47% of the time. It often thought you were on the opposite side of the street.
Inertial + Map (No GPS): Did surprisingly well, getting the sidewalk right 71% of the time. It proved that even without GPS, knowing the rules of the road helps a lot.
The Fusion (GPS + Inertial + Map): This was the champion. It identified the correct sidewalk 80% of the time.

Key Takeaways from the Data:

Big Errors Disappeared: The fused system almost never made huge mistakes (like being 30+ meters off), whereas the other methods did this often.
The "Jaywalking" Rule: The researchers found that if they treated street-crossing as "allowed" (weight 1), the system got confused. But if they treated it as "rare/unlikely" (weight 0.4), the system performed best. It's a balance between being strict and being flexible.
The "Lag" Effect: When turning a corner, the system sometimes "lagged" behind you. This happened because the particles (guesses) were stuck behind a building corner, waiting for the GPS to pull them around the bend. This caused a temporary error in the direction you were walking, but it was a small price to pay for knowing which side of the street you were on.

Conclusion

The paper concludes that by combining a smart step-counter (RoNIN), a map that knows where buildings are, and a shaky GPS signal, you can create a navigation system that is much more reliable than GPS alone.

For a blind pedestrian, this means the difference between being told "Cross the street now" when you are actually standing safely on the sidewalk, versus being told to cross when you are in the middle of traffic. The system doesn't need to be perfect; it just needs to be good enough to know which side of the street you are on, and this method does that significantly better than current technology.

Technical Summary: Accurate Pedestrian Tracking in Urban Canyons

Problem Statement
Pedestrian navigation in "urban canyons"—areas with tall buildings that obstruct satellite signals—suffers from significant degradation in Global Navigation Satellite System (GNSS) accuracy. This limitation is particularly critical for blind or low-vision users who rely on precise localization for wayfinding, such as determining which side of a street they are on. While Visual Positioning Services (VPS) offer an alternative, they are often impractical for continuous use (e.g., when the phone is in a pocket) or for users who cannot visually reference their environment. Furthermore, traditional Kalman filter-based fusion of GNSS and Inertial Measurement Unit (IMU) data often fails when error distributions are non-Gaussian or when GNSS signals are lost.

Methodology
The authors propose a multi-modal fusion approach that combines GNSS data with inertial navigation, utilizing a Particle Filter (PF) to incorporate spatial priors from digital maps. The system operates through three primary components:

Inertial Navigation (RoNIN): The system employs RoNIN (Robust Neural Inertial Navigation), a deep learning-based algorithm that estimates user velocity directly from IMU data (accelerometer and gyroscope) in a fixed world reference frame. This approach is robust to varying phone placements (e.g., hand vs. pocket) and walking styles. However, like all Pedestrian Dead Reckoning (PDR) systems, it is subject to drift over time.
Particle Filtering with Spatial Priors: To mitigate drift and constrain the solution space, the system uses a particle filter with 500 particles. Each particle represents a hypothesis of the user's location $(x, y)$ $(x, y)$ , orientation drift angle $\theta$ $θ$ , and a weight $w$ $w$ .
- Spatial Priors: The filter utilizes a map-based segmentation of the environment into three categories:
  - Impenetrable buildings: Particles landing here are assigned a weight of 0 and are effectively removed.
  - Freely traversable areas (sidewalks/crosswalks): Particles here are assigned a weight of 1.
  - Street surfaces (jaywalking): Particles here are assigned an intermediate weight (experimentally determined as 0.4) to reflect that while pedestrians occasionally cross streets outside crosswalks, they generally do not.
- Resampling: Particles are resampled based on their weights, ensuring the distribution concentrates on plausible paths. The orientation drift component tracks the cumulative error in RoNIN's velocity vector direction.
GNSS-Inertial Fusion: The particle weights are further adjusted based on GNSS data. When the GNSS uncertainty radius is below a threshold, a bivariate Gaussian density centered on the GNSS position is used to update particle weights. Particles closer to the GNSS estimate receive higher weights, effectively pulling the particle distribution toward the GNSS location while respecting the physical constraints of the map.

Experimental Setup
The system was evaluated on six walking routes in downtown San Francisco, including areas with severe GNSS degradation such as the Salesforce Transit Center (a covered hub) and narrow alleys between tall buildings. Data was collected using an iPhone 13, with ground truth established via manual waypoint tapping and assumptions of straight-line travel between points. Performance was measured against three metrics:

Correct Sidewalk Assignment: The proportion of time the estimated location was on the correct sidewalk segment.
Euclidean Error: The distance between the estimated and ground truth locations.
Along- and Across-Street Error: Decomposition of the Euclidean error relative to the street direction.

Key Results
The study compared three configurations: GNSS-only, Inertial-only with Particle Filtering (RoNIN+PF), and Fused (GNSS+RoNIN+PF).

Sidewalk Assignment: The fused approach (GNSS+RoNIN+PF) achieved the highest accuracy, correctly identifying the sidewalk 80% of the time. This significantly outperformed GNSS-only (47%) and RoNIN+PF (71%). The choice of the "jaywalking weight" was critical; a weight of 0.4 yielded the best results, whereas treating streets as fully traversable (weight 1) or fully impassable (weight 0) degraded performance.
Euclidean Error: The fused method produced the lowest median error (11.3 m) compared to GNSS-only (13.6 m) and RoNIN+PF (16.1 m). Crucially, the fused approach drastically reduced the likelihood of large errors; the 90th percentile error was 20.5 m for the fused method, compared to 48.2 m for GNSS-only and 47.4 m for RoNIN+PF.
Directional Error: Consistent with prior literature, GNSS-only was more accurate along the street (6.2 m median) than across it (8.1 m). The fused method reversed this trend, achieving the lowest across-street error (3.1 m) but a slightly higher along-street error (9.0 m) compared to GNSS-only. The authors attribute the increased along-street error in the fused method to a "lag" effect where particles accumulate at corners before turning, a temporary artifact of the particle filter dynamics.

Significance and Claims
The paper demonstrates that combining machine learning-based PDR (RoNIN) with particle filtering and spatial priors offers a robust solution for pedestrian localization in urban canyons where GNSS is unreliable. The authors claim that this approach is particularly valuable for accessibility applications, as it significantly improves the ability to determine which side of the street a user is on—a critical factor for safe navigation.

The work highlights that while inertial-only localization can outperform GNSS in specific metrics (like across-street error) even without GNSS data, the fusion of both modalities provides the most consistent and accurate results, effectively mitigating the drift of dead reckoning and the noise of GNSS. The authors note that while their map segmentation was manual, similar geosegments can be derived automatically from existing GIS data (e.g., OpenStreetMap), suggesting the feasibility of deploying this system in real-world scenarios without extensive manual mapping.