💬 NLP

Mapping the City Through the Lens of Language Models

This paper empirically traces how language models implicitly assume an "ordinary" city by rating anonymized urban profiles across 40 indicators, revealing a shared bias toward larger, faster-growing, and more infrastructure-dense urban forms where typicality and desirability often align.

Original authors: Wanqi Liu, Rong Zhao, Zhizhou Sha, Qinyu Cui, Yecheng Zhang

Published 2026-08-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Wanqi Liu, Rong Zhao, Zhizhou Sha, Qinyu Cui, Yecheng Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are walking into a giant, invisible library where the books are written by a super-smart robot that has read almost everything on the internet. This robot is a "language model," a type of AI that predicts what words come next in a sentence. But here's the tricky part: sometimes, people ask the robot vague questions like, "Tell me about a city." The robot doesn't know which city you mean. Is it a tiny village in the mountains? A sprawling metropolis in the desert? A historic European town? To answer, the robot has to make a guess based on what it has learned. It pulls a "default" city out of its digital hat. The big question is: what does that default city actually look like? Does the robot imagine a city that looks like New York, or maybe one that looks like a generic cartoon town? This matters because if the robot's "average" city is wrong, it might give bad advice to architects, planners, or anyone trying to understand how our world works. It's like if a chef who has never tasted a real pizza decided that the "perfect pizza" was just a plate of cheese and bread, ignoring the sauce and the crust.

This paper is like a detective story where the authors try to figure out exactly what that "default city" looks like inside the robot's brain, without ever naming a real place. They didn't ask the robot, "What is a typical city?" because that would just get a messy, poetic answer full of the robot's favorite words. Instead, they played a clever game. They took real data from thousands of actual cities around the world—things like how big they are, how many people live there, how many roads they have, and how green they are—but they scrubbed all the names and locations off the data. They turned these real cities into anonymous "profiles," like ID cards with numbers but no names. Then, they showed these ID cards to ten different versions of language models and asked, "On a scale of 1 to 7, how much does this look like a 'city' to you?"

The results were fascinating. The robots didn't pick a random city. They consistently rated certain types of cities as the "most typical." The "Default City" in the robot's mind is surprisingly specific. It's a place with a lot of people (about 0.5 million), a huge area of buildings (around 21 square kilometers), and a lot of paved roads and infrastructure. It's not a quiet, sparse town; it's a bustling, dense, and growing place. In fact, the robot thinks a "typical" city is much bigger and more developed than the average city on Earth. Most of the real cities that fit this description are in Europe and North America, while many cities in Africa and parts of Asia were rated as "less typical" by the robots, simply because they are smaller or growing in different ways.

The authors also found that the robots' idea of a "typical" city is closely tied to what they think is a "desirable" city. If a city looks like the robot's ideal version, the robot also thinks it's a good place to live. This suggests that the robot isn't just describing what exists; it's describing what it wants to exist. However, the study is careful to say this isn't a perfect map of reality. The robots are biased toward big, modern, developed places because that's what they've seen most often in their training data. When the researchers adjusted for the size and development level of the cities, the regional differences shrank, but the core idea remained: the robot's "average city" is a large, fast-growing, well-connected urban center.

In the end, the paper shows us that when we ask an AI about "a city," we aren't getting a neutral fact. We are getting a reflection of the robot's own biases and experiences. The "Default City" is a mirror, showing us that these powerful tools tend to imagine the world as a collection of big, modern, developed hubs, often overlooking the diverse, smaller, or differently growing cities that make up the rest of our planet. It's a reminder that even in the digital age, our tools have a perspective, and it's up to us to make sure that perspective includes everyone.

Technical Summary: Mapping the City Through the Lens of Language Models

Problem Statement
Language models (LMs) frequently resolve underspecified queries about "a city" by relying on unstated assumptions regarding urban scale, form, infrastructure, and function. While existing geographic evaluations focus on factual knowledge, task performance, or disparities associated with named locations, they fail to identify the specific "urban profile" that models treat as central to the category when place identity is absent. A direct generation prompt (e.g., "describe a typical city") is insufficient as it conflates category centrality with linguistic style, cultural familiarity, and normative ideals. This paper addresses the need to empirically trace the "Default City"—the behavioral ranking of anonymous urban profiles that models implicitly privilege as the category prototype.

Methodology
The study employs a rigorous, auditable framework to measure model behavior across 40 morphological and functional dimensions of real urban centers, avoiding named locations to prevent bias toward specific cities.

Data Foundation: The authors utilize the GHS Urban Centre Database (R2024A), harmonizing 11,377 quality-controlled morphological centers. A stratified probability sample of 2,000 profiles serves as the primary frame, with a disjoint 512-profile sample for replication. These profiles are mapped across seven conceptual domains: Scale/Extent/Change, Built Form/Land Use, Networks/Mobility/Infrastructure, Ecology/Environment, Population/Housing/Public Life, Economy/Function/Centrality, and Culture/Institutions/Identity.
Behavioral Elicitation: Ten open-weight language model checkpoints (ranging from 7B to 14B parameters, including Qwen, Gemma, Llama, Mistral, and others) rate the typicality of each profile. Instead of open-ended generation, the models provide constrained next-token probabilities for a 1–7 rating scale. The rating is calculated as a probability-weighted sum: $r = \sum k \cdot p(k)$ .
Reliability and Aggregation: The framework implements strict reliability screens, including field-order agreement, response dispersion, and reduced-form agreement checks. Models are grouped by lineage to create dependency components. Aggregation involves medianing values within components and then across components to derive a dependency-balanced consensus.
Analysis Targets: The study evaluates three distinct estimands: city-count typicality, resident-weighted typicality, and region-balanced typicality. It further compares marginal indicator-wise judgments against direct ratings of complete 40-indicator profiles to test behavioral coherence.

Key Contributions

Population-Conditional Formulation: The paper defines "geographic category centrality" not as a static fact but as a behavioral ranking conditional on the model, the elicitation task, and the reference population.
Auditable Multidimensional Framework: It introduces a methodology combining probabilistic judgments, provenance tracking, reliability diagnostics, dependency-aware aggregation, and independent replication to measure latent urban biases without relying on named entities.
Global Behavioral Characterization: It provides the first empirical delineation of the "Default City" portrait across a diverse set of open-weight models, moving beyond task-specific biases to multidimensional judgments of urban form.

Results

The Default City Portrait: The analysis reveals a recurring "scale–growth–infrastructure" core. The typical city favored by these models is characterized by a larger developed area (approx. 21 km²), faster recent population growth, greater mapped infrastructure, higher non-residential capacity, and a less sparse form. These profiles consistently fall in the 81st–90th percentiles of their respective global distributions.
Geographic Affinity: Unadjusted affinity is highest for cities in Europe and Northern America and lowest for Sub-Saharan Africa and Central/Southern Asia. However, this geographic tilt is largely driven by scale and development composition; when adjusting for population, built surface, density, and subnational HDI, the regional disparities shrink significantly, though a residual positive bias for Europe remains.
Model Consistency and Variation: While nine of the ten checkpoints form a relatively compact band of agreement (median divergence 11.1–16.1%), DeepSeek-R1 shows distinct divergence. Agreement is strongest for non-residential capacity, built extent, and infrastructure, but weaker for building-type views and footprint geometry.
Convergence and Desirability: Marginal judgments (indicator-wise) show moderate agreement with direct joint-profile ratings ( $\rho \approx 0.50$ ), suggesting behavioral coherence. Furthermore, where reliably measured, typicality and desirability are closely aligned ( $\rho \approx 0.985$ ), indicating that models often equate what is "typical" with what is "desirable."

Significance
The paper claims to make the vague notion of what models regard as an "ordinary city" empirically traceable. By delineating a shared yet model-dependent portrait, the framework exposes the implicit assumptions language models hold about urban environments. The authors emphasize that these scores are intended for research and audit purposes; because typicality can naturalize planning norms, the results should not be used directly for planning applications without human and local evidence. The study establishes that while models share a core bias toward larger, growing, and well-infrastructure cities, this "Default City" is a behavioral construct contingent on the model's training distribution and the reference population used for evaluation.

Technical Summary: Mapping the City Through the Lens of Language Models

More like this