Simulating Stakeholder Perspectives: Using Multimodal Large Language Models to Support Early-Stage Urban Design
This paper introduces StakeSight, an interactive system leveraging Multimodal Large Language Models and a Retrieval-Augmented Generation pipeline trained on thousands of urban narratives to simulate diverse stakeholder perspectives, thereby supporting more inclusive and efficient early-stage urban design.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Cities are not just collections of buildings and streets; they are living environments shaped by the daily lives of the people who move through them. For decades, urban planners and designers have understood that a successful public space must work for everyone, from the elderly resident walking to the market to the parent pushing a stroller. However, figuring out exactly how different people will experience a new design before it is built has always been a difficult and expensive task. Traditionally, this requires gathering real people for workshops and interviews, a process that takes significant time and money, often failing to capture the voices of those who are most marginalized or hard to reach. As cities grow more complex, the need to understand these diverse human experiences early in the design process has become urgent, yet the methods to do so have remained largely stuck in the past.
To bridge this gap, a team of researchers has developed a new digital tool called StakeSight, which uses advanced artificial intelligence to simulate how different types of people might react to a proposed urban design. The system is built on a foundation of real human stories. The researchers began by collecting and analyzing 3,017 first-person narratives from public media sources, such as journalism archives and podcasts, where people described their actual experiences with cities. By reading these thousands of stories, the team identified five key ways that people perceive their surroundings: their age and life stage, what they are doing in the space, how they move around, their physical and sensory needs, and their personal values or memories attached to a place. These five dimensions form the core of the system, allowing it to understand that a space feels different to a tired commuter than it does to a child playing in a park.
Using this deep understanding of human experience, the researchers built an interactive system where a designer can upload a sketch or photo of a new public space, such as a park or a plaza, and then create a specific "role" to test it against. The designer might choose to simulate the perspective of a 70-year-old woman who uses a wheelchair, or a young parent carrying a heavy grocery bag. Once a role is defined, the system does not simply guess what that person might think. Instead, it searches through its massive database of real human stories to find narratives that closely match the chosen role's characteristics. It then uses a powerful language model to weave these real-world experiences together with the visual design, generating a detailed, first-person response. The output includes a narrative describing what the simulated person sees and feels, a list of specific pros and cons, and a numerical score rating the design on safety, comfort, and accessibility.
The researchers tested this tool with twelve urban designers, architects, and students to see if it could actually help them think more inclusively. The results were encouraging. The participants found the system useful for the early stages of design, noting that it allowed them to explore ideas and spot potential problems before they ever had to organize a real community meeting. One designer described it as a way to "do homework" before meeting real people, helping them ask better questions and prepare for actual engagement. The tool proved particularly effective at bringing to light the needs of groups that are often overlooked, such as people with disabilities or those with specific mobility constraints. For instance, when a designer tested a new bike lane, the system generated feedback from a parent with a stroller who pointed out safety concerns about the narrow path, and from a commuter who valued the efficiency of the route. These conflicting needs, which might have been missed in a standard design review, were highlighted clearly by the simulation.
However, the researchers are careful to note that this tool is not a replacement for real people. The participants in the study were honest about the limitations, observing that while the simulated feedback was often realistic and helpful, it could sometimes feel generic or lack the deep nuance of a real human conversation. The system is designed to be a starting point, a way to lower the barrier to inclusive thinking and help designers consider a wider range of perspectives before they begin the hard work of real-world consultation. It does not claim to know exactly what a community wants, but rather suggests what they might need based on the vast library of human experiences it has learned from. By allowing designers to step into the shoes of others through these simulations, StakeSight offers a new way to imagine cities that are not just visually appealing, but truly welcoming to the diverse lives that will inhabit them.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.