Towards Human-AI Accessibility Mapping in India: VLM-Guided Annotations and POI-Centric Analysis in Chandigarh

This paper presents the adaptation of Project Sidewalk for Chandigarh, India, by integrating VLM-guided annotations to conduct a POI-centric analysis that identified over 1,600 accessibility improvements across 40 km of sidewalks in diverse urban sectors.

The Gist

Imagine you are trying to walk through a city, but you are blindfolded. You don't know if the path ahead is smooth, if there are potholes, or if a curb is too high to step over. Now, imagine doing this for millions of people, including those in wheelchairs or with visual impairments. That is the challenge of urban accessibility.

This paper describes a project called Project Sidewalk, which acts like a "digital detective" to map out these walking paths. Here is the story of how they adapted this tool for the city of Chandigarh, India, using a mix of human volunteers and a smart AI assistant.

1. The Problem: A "One-Size-Fits-All" Tool Didn't Fit

Think of the original Project Sidewalk as a universal remote control designed for American living rooms. It works great in the US, where sidewalks are usually straight, paved, and have standard ramps.

But when they tried to use this "remote" in Chandigarh, India, it didn't work.

• The Mismatch: In India, sidewalks aren't always neat. Sometimes they are just the side of the road, covered in parked scooters, street vendors, or loose bricks. A "missing curb ramp" in the US looks very different from a "makeshift slope" in India.
• The Confusion: If you asked a volunteer to label a "fire hydrant" (common in the US) on an Indian street, they would be confused because that object doesn't exist there. Instead, they need to label "drainage ditches" or "stacked crates."

2. The Solution: Customizing the Toolkit

The team realized they couldn't just copy-paste the American tool. They had to rebuild the toolbox for the Indian context.

• New Labels: They changed the instructions. Instead of asking "Is there a curb ramp?", they asked, "How does the ground transition to the road?" (Is it a smooth slope, a step, or a broken edge?).
• New Pictures: They swapped out photos of American streets for photos of Chandigarh streets so volunteers knew exactly what to look for.

3. The Secret Weapon: The "AI Co-Pilot"

This is the most exciting part. Imagine you are playing a video game, but instead of a static map, you have a smart co-pilot (an AI) who whispers hints in your ear.

• How it works: Before a volunteer starts "walking" down a virtual street, the AI looks at the street type (is it a busy main road or a quiet residential lane?) and the pictures of the street.
• The Hint: The AI says, "Hey, this is a quiet residential street. Don't look for fancy ramps; instead, look for parked scooters blocking the path or loose gravel."
• The Result: This helps the human volunteers focus on the right things, making their job faster and more accurate. The volunteers rated this AI helper 4.66 out of 5, saying it was incredibly useful.

4. The Mission: Mapping the "Heart" of the City

Instead of mapping every single inch of the city (which would take forever), the team focused on Points of Interest (POIs). Think of these as the "destinations" that matter most to people:

• Hospitals (Sector 12)
• Shopping Malls (Sector 34)
• Homes (Sector 45)

They mapped the 1-kilometer walking radius around 230 of these places. It's like drawing a circle around a school and checking if a child with a disability can actually walk safely to the gate.

5. What They Found: The Good, The Bad, and The Ugly

After analyzing 40 kilometers of roads, they found some surprising truths:

• The "Rich" Areas: Commercial areas (shops and malls) generally had the best walking paths. It seems businesses take care of the sidewalks in front of them.
• The "Poor" Areas: Schools, public transport stops, and government offices had the worst accessibility. This is ironic because these are the places where people need to go the most.
• The Hospital Paradox: In the hospital district, the path to the hospital was okay, but the path to the nearby bus stop or food stall was terrible.
• The Big Number: They found 1,644 specific spots where a small fix (like moving a parked bike or fixing a broken brick) could make a huge difference for thousands of people.

The Big Picture

This paper is like a blueprint for a more inclusive city. It shows that by combining human eyes with smart AI, we can quickly find the "broken steps" in our cities.

The ultimate goal isn't just to make a map; it's to give city planners a "to-do list." Instead of guessing where to fix things, they can now see exactly which street needs a ramp or which bus stop needs a clear path, ensuring that everyone, regardless of their ability, can enjoy the city.

Technical Summary

1. Problem Statement

The paper addresses the critical lack of reliable, cost-effective data on sidewalk accessibility in Indian cities, which hinders inclusive urban planning. While tools like Project Sidewalk have successfully mapped accessibility in the US and Europe using crowdsourcing and Google Street View (GSV), direct deployment in India faces significant challenges:

• Contextual Mismatch: Indian street infrastructure differs radically from Western contexts. Sidewalks are often discontinuous, shared with vehicles, or non-existent. Features like standardized ADA curb ramps are rare, replaced by improvised slopes, steps, or drainage channels.
• Annotation Ambiguity: Existing labels and examples confuse annotators in India because they do not reflect local realities (e.g., vendors, parked two-wheelers, and unpaved shoulders).
• Prioritization Gaps: There is a need for a framework to prioritize accessibility improvements based on high-impact locations (Points of Interest) rather than auditing entire road networks indiscriminately.

2. Methodology

The authors adapted Project Sidewalk for Chandigarh, India, through a three-pronged technical approach:

A. Interface and Taxonomy Adaptation

To align the tool with Indian streetscapes, the authors modified the labeling schema:

• Label Restructuring: The original "Curb Ramp" label was replaced with "Curb Style" to capture the heterogeneity of Indian transitions (sloped, stepped, broken, or improvised).
• Tag Localization: Tags were updated to include India-specific obstacles (e.g., street vendors, drainage channels, parked scooters) and removed irrelevant ones (e.g., fire hydrants, mailboxes).
• Visual Localization: All example images and severity scales were replaced with high-resolution images from Chandigarh to reduce cognitive load and improve judgment accuracy.

B. VLM-Guided Mission Guidance

To assist annotators in real-time, the authors integrated a Visual Language Model (VLM) (Google's gemini-2.5-flash) into the workflow:

• Input: The system takes the OpenStreetMap (OSM) road type (e.g., residential, secondary) and the start/end GSV panoramas of a street segment.
• Output: The VLM generates natural language "mission guidance" prompts displayed before the annotator begins.
• Logic: The guidance adapts based on context. For example, on residential roads (often lacking sidewalks), it instructs users to focus on road-surface obstacles and uneven edges. On secondary roads, it directs attention to curb ramps and crosswalks.
• Goal: To reduce cognitive overhead and provide "just-in-time" context without automating the labeling itself.

C. POI-Centric Accessibility Analysis

Instead of auditing all roads, the study focused on a 1-km walkable radius around 230 Points of Interest (POIs) across three distinct land-use sectors in Chandigarh:

• Sector 45: Residential
• Sector 34: Commercial
• Sector 12: Institutional (Hospital area)
• Filtering: Only road segments with at least 75% GSV coverage were included.
• Scoring Metric: An AccessScore was calculated using a weighted vector of labeled features (severity 1–3). Scores were aggregated at three levels:
  1. Segment Level: Normalized score for individual road segments.
  2. POI Level: Length-weighted average of segments within 1km of a POI.
  3. Sector Level: Weighted average across all POI categories in a sector.

3. Key Contributions

1. Contextual Adaptation of Project Sidewalk: The first deployment of the platform in India, featuring a localized interface with revised labels (e.g., "Curb Style"), tags, and visual examples.
2. VLM-Driven Human-in-the-Loop Guidance: A novel integration of a VLM to generate context-aware mission instructions, evaluated with a mean utility score of 4.66/5.
3. Large-Scale POI-Centric Audit: A comprehensive analysis of ~40 km of sidewalks and 230 POIs across three diverse sectors, providing the first granular accessibility baseline for Chandigarh.

4. Results

VLM Guidance Evaluation

Three annotators evaluated 50 street segments based on Relevance, Accuracy, and Usefulness (1–5 Likert scale):

• Relevance: Mean 4.97 (SD 0.26). Annotators found the guidance highly relevant to the visual context.
• Usefulness: Mean 4.61 (SD 0.70).
• Accuracy: Mean 4.40 (SD 0.71).
• Agreement: High inter-annotator agreement on Relevance (Cohen's $\kappa$ = 1.0 between R1 and R3), though some variation existed in Accuracy and Usefulness ratings.

Accessibility Findings (Chandigarh)

• Total Findings: Across 40 km of roads and ~230 POIs, 1,644 locations were identified where infrastructure improvements could enhance accessibility (out of 2,913 total locations).
• Sector Performance:
  • Commercial (Sector 34): Showed the best overall accessibility scores.
  • Institutional (Sector 12): Healthcare facilities were relatively accessible, but surrounding amenities (transit stops, food joints) lagged significantly.
  • Residential (Sector 45): Moderate scores, with specific gaps in utilities and public services.
• Critical Gaps:
  • Education and Transport: These categories had the lowest accessibility scores across all sectors, indicating a need for prioritization.
  • Long Negative Tail: The distribution of segment scores showed a "long negative tail," indicating that a significant number of road segments have severe accessibility barriers.
  • Functional vs. Physical: While major institutions (hospitals) were accessible, the "last mile" connectivity to daily life amenities (transit, food) was poor.

5. Significance

• Scalability for the Global South: This work demonstrates a replicable framework for adapting Western-centric accessibility tools to the Global South, where infrastructure is often informal and heterogeneous.
• AI-Augmented Crowdsourcing: It validates the use of VLMs not just as auto-labelers, but as contextual assistants that improve human annotator efficiency and consistency in complex, non-standard environments.
• Policy Impact: By shifting the focus to POI-centric analysis, the study provides actionable data for urban planners to prioritize investments where they yield the highest return on investment (e.g., improving access to schools and transit stops rather than just general road repair).
• Baseline Data: The study establishes the first detailed, geolocated accessibility dataset for Chandigarh, a city known for its planned grid layout, serving as a model for other Indian cities.