💻 computer science

When to Ask a Question: Understanding Communication Strategies in Generative AI Tools

This paper proposes a theoretical framework and empirical analysis for optimizing the trade-off between user burden and preference accuracy in generative AI, demonstrating that strategically soliciting information can effectively mitigate systematic biases inherent in model-based inference while maintaining efficiency.

Original authors: Charlotte Park, Kate Donahue, Manish Raghavan

Published 2026-05-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Charlotte Park, Kate Donahue, Manish Raghavan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a chef (the AI) and a customer (the user) walks into your kitchen. The customer says, "I want a sandwich."

In the old days, a computer system might just guess what you want based on what most people order. If 90% of people like ham, the chef makes a ham sandwich. If you actually wanted a vegan wrap, you'd get a ham sandwich anyway. You'd eat it because it's "good enough," but it wasn't your perfect sandwich.

This paper asks a simple but tricky question: Should the chef stop and ask, "Do you want ham or a wrap?" before starting to cook?

Asking questions has a cost: it takes time and effort for the customer to answer. But not asking has a cost too: the chef might guess wrong and make a sandwich you hate.

Here is the breakdown of the paper's findings using everyday analogies:

1. The "Guessing Game" Problem

Generative AI (like the chatbots you use) is flexible. You can give it a tiny prompt ("Write an email") or a huge one ("Write a formal email to my boss about a delay, using a polite tone").

The Risk: If you don't give enough details, the AI has to fill in the blanks. It usually fills them with what "average" people like. This is like the chef assuming everyone likes ham. If you are an outlier (someone who hates ham), the AI's "best guess" will be wrong for you.
The Solution: The AI could ask you questions to clarify. But if it asks too many, you get annoyed and just want the food fast.

2. The "Noise" Factor

The paper uses a model where user preferences are like a coin flip.

Low Noise (Clear Signal): If the AI asks, "Do you like spicy food?" and you say "Yes," it's a very strong clue. The AI can probably guess the rest of your preferences correctly. Verdict: Asking is worth it.
High Noise (Static): If the AI asks, "Do you like spicy food?" and your answer is random or confusing, the AI learns nothing new. Asking just wastes your time. Verdict: Don't ask; just guess based on the average.

3. The "Majority Rule" Trap

The paper shows that if the AI only cares about making the most people happy (the "Utilitarian" goal), it often ignores the minority.

The Scenario: Imagine 90% of people want a "Budget Trip" and 10% want a "Luxury Trip."
The AI's Move: If the AI doesn't ask questions, it will plan a Budget Trip for everyone. The 10% who wanted luxury get a bad experience.
The Fix: If the AI is programmed to care about fairness (making sure the minority isn't ignored), it will ask more questions to figure out who is who. It realizes that asking a few questions helps it stop treating the "Luxury" people like "Budget" people.

4. The Surprising Twist: Fairness Doesn't Always Mean More Questions

You might think, "If we want to be fair, the AI should ask everyone every possible question to get it perfect."

The Reality: The paper finds this isn't true.
- If the AI cares too much about fairness (trying to protect the worst-off person), it might realize that asking questions is too expensive for the user.
- Instead, it might decide to just "hedge its bets." It might make a generic output that isn't perfect for anyone but isn't terrible for anyone, rather than spending the user's time asking questions that might still get it wrong.
- Analogy: It's like a teacher deciding whether to grade every student's homework individually (fair but slow) or just give the whole class a standard grade (fast but maybe unfair). Sometimes, the "fair" choice is to stop grading individually and just give a standard result because the cost of checking is too high.

5. The "Diversity" Paradox

The paper also looked at whether asking questions makes the AI's output more diverse (less "homogenized").

The Logic: If the AI asks questions, it learns more about you, so it can make a unique sandwich just for you. This increases diversity.
The Catch: If the AI is trying to be super fair, it might stop asking questions (as mentioned above). If it stops asking, it goes back to guessing the "average" sandwich.
Result: Being obsessed with fairness can sometimes accidentally make the AI's output less diverse because it stops gathering the information needed to be unique.

Summary

The paper builds a mathematical model to figure out the "Goldilocks" zone for AI:

When to ask: Ask when the answers are likely to be clear and helpful (low noise).
When to stop: Stop asking when the answers are likely to be random noise or when the cost of asking is too high for the user.
The Fairness Balance: If you want the AI to be fair to minority groups, it usually needs to ask more questions. But if you push for fairness too hard, the AI might stop asking questions entirely and just guess, which can ironically make things less diverse.

The core message is that asking questions is a trade-off. It's a balancing act between saving the user time and making sure the AI doesn't just guess based on what the "majority" likes.

Technical Summary: When to Ask a Question in Generative AI

Problem Statement

Generative AI models, particularly Large Language Models (LLMs), differ from traditional recommender systems by allowing users to provide inputs with varying degrees of specificity. This flexibility often leads users to omit details, forcing models to infer missing information based on population-level distributional knowledge. While this inference offers convenience, it risks privileging majority viewpoints and disadvantaging users with atypical preferences, leading to algorithmic homogenization and fairness concerns.

Unlike traditional systems where feedback is often implicit (e.g., clicks), generative AI enables explicit interaction. LLMs can actively solicit missing information through natural language queries. However, a fundamental trade-off exists: while eliciting more information improves personalization and fairness, it imposes a burden on users who value efficiency. The central problem addressed is determining when and how much information an AI system should solicit to balance user burden against preference representation, specifically analyzing the impact on utility, fairness, and output diversity.

Methodology

The authors develop a stylized theoretical model of user-LLM interaction to formalize this trade-off.

1. Preference Generation Model

User preferences are modeled as a vector of $n$ binary features. The population is divided into two clusters (Cluster 0 and Cluster 1) with opposing base preferences (all 0s or all 1s).

Cluster Assignment: Users are assigned to Cluster 0 with probability $\alpha$ and Cluster 1 with probability $1-\alpha$ .
Noise: Each feature is independently flipped with probability $p < 0.5$ .
Correlation: This structure captures the intuition that preferences are correlated; knowing a user's preference on one dimension aids in inferring others.

2. Interaction Protocol

Initial State: The user provides an initial prompt, but the LLM does not know the user's full preference vector.
Querying: The LLM chooses to query $k$ features ( $0 \le k < n$ ) directly from the user.
Inference: Based on the $k$ revealed bits and the known distribution parameters ( $\alpha, p$ ), the LLM infers the remaining $n-k$ features to generate an output.
Utility Function: User utility is defined as $U(k, B) = n - k - B$ , where $k$ is the number of questions asked (cost) and $B$ is the number of incorrectly guessed features (disagreement).

3. Optimization Objectives

The paper analyzes two distinct optimization goals:

Utilitarian Welfare ( $\gamma = 0$ ): Maximizing the expected utility across the population.
Inequality-Averse Welfare ( $\gamma > 0$ ): Maximizing a social welfare function that incorporates aversion to inequality (e.g., Nash welfare or maximin fairness). The parameter $\gamma$ controls the trade-off between equity and efficiency.

Key Contributions and Results

1. Optimal Response Policy (Inference)

Utilitarian Case: The optimal strategy is to identify the user's likely cluster based on revealed bits and guess the remaining features to match that cluster.
Fairness-Aware Case: As inequality aversion ( $\gamma$ ) increases, the optimal response policy shifts from catering to the majority to "hedging." The LLM produces mixed outputs that balance the preferences of different clusters rather than strictly following the most probable inference. Under maximin fairness ( $\gamma \to \infty$ ), the policy becomes highly sensitive to the specific configuration of revealed bits to avoid worst-case outcomes for minority users.

2. Optimal Querying Policy (Information Elicitation)

Utilitarian Trade-off: The optimal number of queries $k$ is non-monotonic with respect to noise ( $p$ ). If the signal is too noisy ( $p \approx 0.5$ ), the cost of asking questions outweighs the benefit of better inference, leading to $k=0$ . Conversely, with reliable signals, asking questions improves utility.
Majority Bias: Under pure utilitarian optimization, if one cluster is significantly larger (e.g., $\alpha$ is close to 1), the model optimally chooses $k=0$ even for minority users, as the expected gain from clarifying a small group does not offset the aggregate cost. This systematically underserves minority preferences.
Fairness-Driven Elicitation: When optimizing for inequality-averse welfare, the optimal $k$ changes. Initially, as $\gamma$ increases, the model asks more questions to better distinguish minority users. However, for very high $\gamma$ (maximin), the optimal strategy may revert to asking fewer questions or none, depending on the noise level, because the cost of querying may outweigh the marginal utility gain in a worst-case scenario.

3. Diversity and Homogeneity

The paper challenges the intuition that fairness always leads to higher output diversity.

Non-Monotonicity: The entropy of the LLM's output distribution is non-monotonic with respect to $\gamma$ .
Mechanism: Moderate levels of inequality aversion may increase diversity by prompting the model to ask more questions, thereby gathering more information to generate varied outputs. However, at extreme levels of aversion, the model may stop soliciting information (to avoid user burden or due to diminishing returns), leading to a deterministic "safe" guess that reduces output diversity. Thus, the relationship between fairness and diversity is mediated by the model's information-soliciting behavior.

4. Empirical Validation

Using a sushi preference dataset (5,000 users ranking 10 items), the authors validated the theoretical predictions:

Welfare vs. Queries: As the inequality aversion parameter $\gamma$ increased, the optimal number of queries $k$ increased (from 4 to 10 in the experiment), shifting from a utilitarian balance to full preference elicitation.
Fairness-Utility Frontier: Increasing $k$ reduced the Gini coefficient (inequality) but decreased mean utility due to query costs. The optimal $k$ depends entirely on the chosen $\gamma$ .

Significance and Claims

The paper claims to provide a formal framework for understanding the trade-offs in interactive generative AI. Its primary significance lies in demonstrating that:

Information Elicitation as a Fairness Tool: Strategic questioning can mitigate systematic biases inherent in preference inference, allowing generative tools to better incorporate diverse perspectives.
Context-Dependent Strategies: There is no universal "best" number of questions. The optimal strategy depends on the noise level of user data, the size of preference clusters, and the system's objective function regarding fairness.
Complexity of Diversity: Optimizing for fairness does not automatically guarantee diverse outputs; the relationship is mediated by how much information the system chooses to solicit.

The authors maintain modesty regarding their claims, acknowledging that their model relies on a highly stylized Bernoulli mixture of preferences and assumes the LLM has perfect knowledge of the population distribution. They position this work as a first step toward understanding the relationship between preference inference, information acquisition, and fairness, suggesting that future work should explore more complex preference structures and differential costs for information.