The Geography of Algorithmic Judgment: LLM Intermediaries, Place Identity, and Racial Steering in Housing Search

Technical Summary: The Geography of Algorithmic Judgment

Problem Statement
Large Language Models (LLMs) are increasingly functioning as intermediaries in housing searches, mediating access to information and recommendations through conversational interfaces. While these tools offer efficiency, there is a critical concern that they may amplify historical patterns of segregation and discrimination present in their training data. Specifically, the paper addresses the risk of "racial steering," where algorithms systematically guide users of specific racial identities toward or away from certain neighborhoods, potentially violating the Fair Housing Act (FHA). Unlike previous studies that often rely on static demographic cues, this research investigates whether steering is an emergent behavior resulting from the complex interaction between a user's identity, their articulated lifestyle preferences, and the model's internalized spatial logic regarding specific urban environments.

Methodology
The authors conducted a behavioral audit of seven LLMs (including closed-source models like GPT-4o, Claude Sonnet, and Gemini 2.0 Flash, and open-weight models like Llama 3.1 and DeepSeek V3) across four major U.S. cities: Chicago, Houston, New York City, and Los Angeles.

• Experimental Design: The study utilized a paired-testing methodology adapted from fair housing enforcement, employing iterative prompting across three conditions:
  • P0 (Baseline): Recommendations based solely on demographic cues (Black, Hispanic, White, or Neutral/Unspecified).
  • P1 (Preference-Conditioned): Recommendations incorporating specific lifestyle preferences (e.g., budget constraints, family needs, safety, walkability).
  • P2 (Inferred Priorities): Recommendations where the model is asked to infer user priorities based on stated preferences and act on them.
• Data Collection: The dataset comprised 20,160 data points (2,880 per model), with each run generating five recommended zip codes and justification keywords.
• Analysis Metrics:
  • Probability of Recommendation (PoR): Calculated as the proportion of times a specific zip code was recommended for a given identity.
  • Spearman's Correlation: Measured the relationship between the percentage of a specific racial group in a zip code and the PoR for that group.
  • Opportunity Index: A composite z-score measure based on seven census variables (income, poverty, homeownership, etc.) to assess the material access to opportunity in recommended areas.
  • Point Pattern Analysis: Used to account for spatial spillover effects between zip codes.

Key Contributions

1. Behavioral Audit of Multiple Models: The study provides a comparative analysis of seven distinct LLMs, moving beyond single-model evaluations to assess systemic trends in housing recommendations.
2. Contextualized Prompting: By moving beyond simple demographic cues to include iterative, preference-based prompting, the study demonstrates how steering is mediated through the interpretation of user identity rather than just demographic signaling.
3. Geographic Specificity: The research challenges the assumption that LLM evaluation results are generalizable across cities, highlighting how local urban histories and socioeconomic characteristics significantly influence steering patterns.
4. Mitigation Testing: The paper evaluates the effectiveness of prompt-level interventions (adding context or asking for inference) in mitigating bias, finding that these strategies yield uneven results depending on the demographic group and city.

Results

• Emergent Steering: Steering was found to be an emergent behavior of the model's interpretive license rather than a static property. While present at baseline (P0), the addition of preference context (P1 and P2) often increased or reconfigured the number of models exhibiting steering behaviors.
• Demographic Variance:
  • Black Test Cases: Generally experienced "affirmative steering" toward Black-majority neighborhoods across all cities and conditions, though the magnitude varied. In Houston, preference conditions (P1/P2) notably reversed or eliminated this steering.
  • White Test Cases: Were consistently steered toward White-majority neighborhoods with the least fluctuation in direction.
  • Hispanic Test Cases: Exhibited the most variable behavior, with steering direction and magnitude shifting significantly across cities and prompt conditions.
• City-Specific Patterns:
  • Los Angeles: Showed the highest magnitude of steering for Black test cases (e.g., Mistral at 90%).
  • Houston: Displayed irregular behaviors, including the reversal of steering directions for Black and Hispanic groups when preferences were introduced.
  • Chicago: Family-based prompts (PF3) led models like GPT-4o and Claude to recommend Black users to South Side zip codes with lower opportunity indices, while White and Hispanic users with identical preferences were recommended to North Side zip codes with higher opportunity indices.
• Inference Prompting (P2): Requiring the model to infer priorities produced uneven results. For Black users, it often reduced steering magnitude; for Hispanic users, it sometimes eliminated or reversed steering; for White users, results were noisy and city-dependent.

Significance and Claims
The paper argues that steering in LLMs is not merely a function of demographic bias but a result of how models interpret what specific desires (e.g., "safety," "community") mean for a user of a particular demographic within a specific urban geography. The authors claim that:

• The City is Not a Neutral Unit: Results from one local market cannot be assumed to generalize to another. Local expertise is required to evaluate AI tools in place-based sectors because steering emerges from the interaction of user identity and local spatial logic.
• Interpretive License: The "interpretive license" models hold—how they translate a preference into a location—is the primary mechanism for steering. This suggests that models may interpret the same housing preference differently depending on the racial identity of the user.
• Risk to Fair Housing: As LLMs become infrastructure for housing search, they risk reenacting spatial maps of inequality and limiting user visibility to alternatives the model deems "suitable."
• Need for Domain Judgment: The authors conclude that without deliberate intervention and critical judgment from real estate practitioners, LLMs may become a new mechanism for reproducing inequitable patterns. They emphasize that legal and institutional commitments to fair housing require domain-specific expertise to ensure AI tools do not undermine these goals.