💬 NLP

Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Research

This paper proposes Situated Interaction Auditing (SIA), a user-centered framework that addresses the limitations of traditional third-person audits by investigating how LLMs systematically alter their responses based on user profile signals such as gender and socioeconomic status during personal interactions.

Original authors: Andrés Abeliuk, Cinthia Sanchez Macias, Valentina Alarcón, Álvaro Madariaga, Claudia Lopez

Published 2026-06-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Andrés Abeliuk, Cinthia Sanchez Macias, Valentina Alarcón, Álvaro Madariaga, Claudia Lopez

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are walking into a very smart, well-read library where a single librarian (the AI) helps you with any question you have.

For years, researchers have tested this librarian by asking them to write stories about other people. They would ask, "Write a story about a nurse named Kelly," and then, "Write a story about a nurse named Joseph." They checked if the librarian described Kelly as "warm and friendly" but Joseph as a "natural leader." This is like checking if the librarian is biased when talking about strangers.

The Problem: The Librarian Misses the Person Standing in Front of Them
This paper argues that this way of testing is missing a huge part of the picture. In real life, you don't just ask the librarian to write about strangers; you ask them to help you.

The authors say that when you ask the librarian for advice, the librarian doesn't just look at your question; they look at who you are. They might guess your background, your social status, or your gender based on your name, the way you speak, or your accent.

The paper calls the old way of testing "Third-Person Audits" (judging how the AI talks about others) and proposes a new way called Situated Interaction Auditing (SIA). This is like judging the librarian based on how they treat you specifically.

The New Experiment: The "Name Game"
To prove this, the researchers ran a test in Chile. They didn't change the questions they asked the AI. They kept the questions exactly the same, like: "How do I negotiate a salary raise?" or "How do I fix this computer error?"

But, they changed the "name tag" the AI saw before answering.

Group A: Used names and surnames that signal a high social status (often associated with wealth and elite families in Chile).
Group B: Used names and surnames that signal a low social status (often associated with working-class or indigenous backgrounds in Chile).

What They Found: The "Patronizing" vs. "Empowering" Gap
Even though the questions were identical, the answers were different depending on the name tag:

The Tone Shift: When the AI thought it was talking to a high-status person, it gave advice that was direct, confident, and used "power words" (like "network," "capacity," "strategy"). It sounded like a peer giving a pep talk.
The Patronizing Shift: When the AI thought it was talking to a low-status person, it gave advice that was more hesitant, emotional, and "safe." It used words like "maybe," "you might consider," and focused on "quality" or "overcoming struggles."

It's as if the librarian told the high-status person, "Here is your strategy to win," but told the low-status person, "Here is some gentle encouragement to try your best." The low-status user wasn't getting the same sharp, actionable tools; they were getting a softer, more tentative version of the answer.

Why This Matters
The paper claims that current AI safety tests are blind to this. They check if the AI is mean to a character in a story, but they don't check if the AI is treating you differently based on your identity.

The Analogy of the "View from Nowhere"
The authors compare old testing to a "view from nowhere"—pretending the user is a blank, invisible ghost. They argue that in reality, the user is a real person with a history, a name, and a social standing. The AI is reacting to that person, not just the text on the screen.

Summary of the Paper's Claims

The Blind Spot: We are testing AI bias by looking at how it describes others, but we are ignoring how it treats us.
The Method: They created a new framework (SIA) to measure how AI changes its tone, vocabulary, and advice based on user signals like names and writing style.
The Evidence: In their Chilean case study, they showed that AI gives "high-status" users more assertive, technical advice, while giving "low-status" users more tentative, emotionally supportive advice for the exact same questions.
The Goal: They want the AI community to start testing models by simulating real conversations with different types of people, not just by asking the AI to write stories about fictional characters.

The paper does not claim to have fixed this problem yet, nor does it offer a specific software patch. It simply argues that we need to change how we look for bias to include how the AI treats the person asking the question.

Technical Summary: Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Research

1. Problem Statement

Current research on bias in Large Language Models (LLMs) is dominated by third-person audits. These evaluations treat the LLM as an external observer or decision-maker, assessing how the model represents or evaluates demographic groups (e.g., generating a resume for a specific gender or summarizing a story about a specific nationality). This paradigm assumes a neutral, undifferentiated user whose identity is irrelevant to the interaction.

However, this approach creates a structural blind spot: in real-world Human-AI Interaction (HAII), the user is an active participant, not an external subject. LLMs are increasingly used in open-ended, personal interactions (e.g., career advice, health guidance, debugging) where the model implicitly constructs a representation of the user based on input signals. Consequently, bias may not manifest in how the model describes others, but in how it treats the interlocutor based on perceived identity. Existing benchmarks fail to capture these "first-person" biases, where identical requests yield systematically different responses depending on who is asking.

2. Methodology: Situated Interaction Auditing (SIA)

The authors propose Situated Interaction Auditing (SIA), a framework designed to study how user-profile signals shape LLM response quality, content, and tone.

2.1 User-Profile Signal Types

SIA defines three types of signals that convey user identity, ranging from implicit to explicit:

S1 (Implicit Sociodemographic Markers): Surface features inferred by the model without explicit self-identification, such as names (carrying ethno-racial and class associations) or geographic references.
S2 (Writing Style and Dialect): Linguistic forms including register, formality, dialectal features, and grammatical patterns (e.g., Guaraní-inflected Spanish or AAVE) that signal regional and social identity.
S3 (Stated Identity): Explicit self-descriptions embedded in the prompt (e.g., "I am gay," "I have a diagnosis").

2.2 Design Principles

To ensure ecological validity and rigor, SIA studies must adhere to five principles:

Semantic Equivalence: Prompt variants must be word-for-word identical except for the user-profile signal.
Ecological Validity: Signals must reflect naturally occurring language rather than constructed extremes.
Multi-Signal Analysis: Signals should be studied in isolation and combination (e.g., gender $\times$ socioeconomic status) to capture intersectional effects.
Realistic Interaction Design: Audits should reflect real-world usage patterns, ideally extending beyond single-turn exchanges.
Diverse Outcome Measurement: Bias in open-ended interactions is continuous, not binary. Metrics must capture gradations in quality, tone, and content.

2.3 Outcome Measurement

SIA operationalizes bias detection through three families of metrics:

Lexical Quality: Vocabulary diversity (type-token ratio), structural elaboration (syntactic tree depth), and response length.
Stance and Framing: The ratio of hedges (tentative language) to boosters (assertive language), agency attribution, and sentiment analysis.
Content Coverage: The presence or absence of specific informational elements, measured via LLM-as-judge evaluation against task-specific checklists and the frequency of stereotyped role attributions (via odds ratios of specific nouns/adjectives).

3. Case Study: Empirical Validation

To ground the framework, the authors conducted a controlled audit in the Latin American (Chilean) context.

Design: A matched-guise experiment using 118 user profiles constructed from names and surnames validated for demographic associations in Chile. The study crossed Gender (Male/Female) and Socioeconomic Status (SES) (High/Low).
Tasks: 27 prompts across 9 domains (e.g., Employment, Technology, Health) asking for "how-to" advice.
Models: GPT-4o mini and Qwen-2.5-7B-instruct.
Procedure: Identical prompts were injected with different user profiles as system messages. Responses were analyzed using the SIA metric families.

Key Results

Domain-Specific Bias: Aggregate effects were small, but significant differential treatment emerged in specific domains, particularly Employment.
High-Status vs. Low-Status Treatment (GPT-4o mini):
- Tone: High-status profiles received responses with significantly higher positive sentiment and fewer hedges. Low-status profiles received responses with higher neutral sentiment, increased hedging, and more boosters, suggesting a patronizing or overly cautious register.
- Content: High-status responses utilized action-oriented and network-focused vocabulary (e.g., "network," "capacity," "personalization"). Low-status responses relied on affectively warm but operationally generic terms (e.g., "quality," "exceptional," "success").
- Technicality: Contrary to the "dumbing down" hypothesis, the model assigned higher technicality scores to low-status profiles in the Employment domain, yet the stance remained tentative.
Model Dependence: The Qwen model replicated the finding that SES signals trigger differential treatment but inverted the direction of sentiment (assigning higher positive sentiment to low-status profiles), indicating that while the phenomenon of bias is robust, its specific manifestation depends on model training.
Intersectionality: The study demonstrated that gender and SES signals interact, producing unique bias patterns that single-dimension audits would miss.

4. Key Contributions

Framework Proposal: Introduction of Situated Interaction Auditing (SIA), shifting the focus from third-party representation to first-person interactional bias.
Typology and Principles: Formalization of user-profile signal types (S1–S3) and five design principles for conducting rigorous situated audits.
Empirical Evidence: A pilot study demonstrating that user-profile signals (specifically name-based gender and SES) systematically alter LLM responses in lexical, agentive, and stereotyping dimensions, invisible to traditional third-person audits.

5. Significance and Research Agenda

The paper argues that SIA represents a necessary evolution in NLP research to close the gap between technical evaluation and real-world deployment.

New Evaluation Norms: The authors propose profile-conditioned assessment, where model performance is scored not just on absolute quality but on consistency across user-profile perturbations. This challenges the "neutral user" assumption inherent in current leaderboards.
Dataset Requirements: Future work requires naturalistic interaction corpora with demographic metadata and culturally grounded signal lexicons, as English-centric datasets cannot be directly translated to other linguistic contexts.
Methodological Bridges: SIA calls for integrating methods from sociolinguistics (matched guise techniques), HCI (participatory design), and sociology to better understand intersectional effects.
Open Problems: The paper identifies causal identification (distinguishing pretraining vs. instruction-tuning effects), signal interaction effects (how models handle conflicting signals), and mitigation strategies (balancing fairness with context sensitivity) as critical areas for future research.

The authors conclude that while their case study is limited to a single-turn design and specific cultural context, it serves as proof-of-concept that first-person interaction bias is real, detectable, and requires a new paradigm for evaluation.