💻 computer science

How Many Visual Levers Drive Urban Perception? Interventional Counterfactuals via Multiple Localised Edits

This paper proposes a lever-based interventional counterfactual framework that uses structured, localized image editing to identify which specific visual elements—such as mobility infrastructure or physical maintenance—most significantly influence human perceptions of urban safety.

Original authors: Jason Tang, Stephen Law

Published 2026-04-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jason Tang, Stephen Law

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The "Urban Remote Control": Making Cities Feel Safer, One Click at a Time

Imagine you are walking through a neighborhood at night. You feel a sudden prickle of unease. Is it because the streetlights are dim? Is it because there’s graffiti on the walls? Or is it because the sidewalk is cracked and messy?

If you could grab a magic remote control for the city, which button would you press to make that street feel instantly safer? Would you add a flowerbed, repaint the crosswalk, or fix a broken storefront?

That is the question researchers at University College London (UCL) are trying to answer.

The Problem: The "Black Box" of City Feelings

Right now, we have AI models that can look at a photo of a street and say, "This place feels unsafe" or "This place feels wealthy." These models are good at spotting patterns, but they are terrible at explaining why.

It’s like having a friend who says, "I don't like this restaurant," but when you ask why, they just shrug. They can tell you what they feel, but they can't tell you exactly what needs to change to make them like it.

The Solution: The "Lever" Framework

The researchers created a way to turn that "shrug" into a precise list of instructions. Instead of just looking at pixels, they created "Visual Levers."

Think of these levers like the settings on a soundboard in a music studio. Each lever controls one specific "vibe" of the city:

The Maintenance Lever: Fix the graffiti, clean the litter, or repair the walls.
The Greenery Lever: Add some plants or trees.
The Infrastructure Lever: Repaint the lines on the road or fix the crosswalks.

How the "Magic Remote" Works (The Process)

The researchers built a three-step digital factory to test these levers:

The Architect (The Planner): The AI looks at a real photo and says, "Okay, in this specific street, I could try to 'pull the lever' to add greenery near that entrance or remove that trash pile."
The Artist (The Generator): An AI artist takes the original photo and "edits" it based on those ideas. It’s like using Photoshop, but instead of a human doing it, the AI tries to "imagine" what the street would look like if the lever were pulled.
The Inspector (The Auditor): This is the most important part. Because AI can sometimes get "hallucinations" (like accidentally turning a tree into a giant purple monster), a second AI acts as a strict inspector. It checks: "Did we actually just add greenery, or did we accidentally change the whole street? Does this look real, or does it look like a cartoon?" If the edit fails the inspection, it gets thrown in the trash.

What Did They Find? (The Results)

They tested this on 50 different scenes from cities like San Francisco, Singapore, and Amsterdam. Even though they used an AI "proxy" to judge safety rather than real humans (which is their next step), they found some fascinating patterns:

The "Clean Up" Effect: Fixing physical things—like repairing a building's face or cleaning up litter—consistently made the AI think the street looked safer.
The "Roadway" Boost: Surprisingly, the biggest "safety boost" came from Mobility Infrastructure. Simply repainting lane markings or crosswalks made the streets look much more "orderly" and safe.
The "Greenery" Paradox: Adding plants made things look better, but trying to "manage" trees (like pruning them) actually made the AI think the street looked less safe. This might be because the AI thinks "more leaves = more hiding spots," even if a human would see it as "neatly trimmed bushes."

Why Does This Matter?

This isn't just about playing with photos. This research is a blueprint for the future of urban planning.

Instead of city officials guessing, "Maybe we should plant more trees?" they could use these "levers" to run simulations. They could ask: "If we spend \ $10,000 on repainting crosswalks versus \$ 10,000 on cleaning graffiti, which one will actually make our citizens feel more secure walking home at night?"

It’s about moving from guessing how a city feels to engineering how a city feels.

Technical Summary: How Many Visual Levers Drive Urban Perception?

1. Problem Statement

Current street-view perception models (which predict subjective attributes like safety, wealth, or liveliness) are primarily correlational. While they can assign a score to a scene, they fail to provide mechanistic explanations. Specifically, they cannot identify which localized, plausible visual changes would actually shift a human's judgment for a specific scene. Existing explainability methods (like SHAP or saliency maps) identify where a model looks, but they do not test what happens if those features are manipulated. Furthermore, urban design research suggests many cues affect safety, but there is no framework to test these cues through scene-specific, validly-audited interventions.

2. Methodology

The authors propose an interventional counterfactual framework that treats explainability as a bounded search over structured "lever" edits. The process is divided into three distinct phases:

Phase 1: Lever Candidate Construction: The authors define a "lever" as a tuple $l = (c, s, d, \tau)$ , representing a concept (e.g., greenery), spatial support (the specific area in the image), direction (add, remove, or repair), and a constrained edit template. A Vision-Language Model (VLM) acts as a planner to ground these concepts into specific scene regions.
Phase 2: Bounded Stochastic Intervention & Validity Audit:
- Generation: Using prompt-conditioned diffusion models (e.g., Qwen-Image-Edit), the system attempts to generate the edit.
- Auditing: Because generative models often introduce unintended "drift," every edit must pass a VLM-based validity audit. An edit is only retained if it satisfies four criteria: (1) same-place preservation, (2) locality to the target, (3) realism, and (4) plausibility of the intended lever.
Phase 3: Model-based Evaluation: The retained "valid" edits are passed through a pre-trained perception model (ViT-B/16) to calculate a proxy score shift ( $\Delta_i$ ). This identifies which levers are "promising" candidates for further human validation.

3. Key Contributions

A New Explanatory Unit: The introduction of the lever-based interventional counterfactual formulation, moving beyond pixel-level attribution to semantic, actionable interventions.
Generation-and-Audit Pipeline: A scalable workflow that combines generative AI with a VLM "critic" to ensure that counterfactuals are not just mathematically possible, but visually and semantically valid.
Pilot Benchmark: A 50-scene multi-city dataset that provides a diagnostic look at how well current generative tools can perform urban interventions.

4. Key Results

The pilot study across five cities (Amsterdam, Abuja, San Francisco, Santiago, and Singapore) yielded several preliminary findings:

Effectiveness of Lever Families:
- Mobility Infrastructure (e.g., repainting crosswalks/lane markings) showed the largest mean auxiliary safety shift ( $+0.579$ ).
- Physical Maintenance (e.g., facade repair, graffiti removal) was the most consistent positive family ( $+0.344$ ).
- Environmental Amenity was heterogeneous: adding greenery was highly positive ( $+0.650$ ), but managing tree canopies was negative, likely due to the proxy model's bias toward "more greenery = safer" regardless of sightlines.
Feasibility & Coverage: The system successfully generated 177 valid edits from 250 candidates. However, certain interventions like lighting repair proved difficult for current prompt-only editors (low validity rate).
City Variations: San Francisco and Santiago showed the highest number of "promising" levers per scene, while Abuja and Singapore showed minimal or negative shifts.
Baseline Correlation: Interestingly, there was no significant correlation between a scene's initial safety score and the number of valid levers it admitted, suggesting "editability" depends more on scene geometry than initial safety levels.

5. Significance

This work represents a shift in Explainable AI (XAI) from post-hoc attribution (explaining what the model did) to interventional explanation (explaining how the world could be different). By enforcing a strict validity audit, the authors ensure that the "explanations" provided are grounded in reality rather than generative artifacts. While the current results use a model-based proxy, the framework provides a structured pipeline to eventually use human pairwise judgment as the ultimate ground truth, offering a powerful tool for urban planners to test how specific design interventions might improve perceived safety and liveliness.