Algorithmic Feature Highlighting for Human-AI Decision-Making

This paper investigates algorithms that select a small subset of case-specific features to assist human decision-makers, demonstrating that while optimizing for "sophisticated" agents who understand the selection process is computationally difficult, optimizing for "naive" agents is tractable and provides a more robust framework for human-AI complementarity.

The Gist

Imagine you are a doctor in a busy emergency room. A patient is rushed in, and their medical file is 500 pages long. You don’t have time to read it all. Instead, an AI assistant scans the file and hands you a small sticky note with just three pieces of information: Heart rate, Blood pressure, and Temperature.

This paper, written by researchers at Stanford, explores exactly how that "sticky note" should be designed to help you make the best decision.

The Core Problem: The "Information Diet"

In many high-stakes jobs—hiring managers, judges, doctors, or real estate appraisers—we use AI. But we don't always want the AI to make the final choice; we want it to highlight the most important parts so we can decide.

The problem is that humans have "limited bandwidth." We can only process so much at once. If the AI gives us too much, we get overwhelmed. If it gives us too little, we miss the big picture. The researchers wanted to find the "Goldilocks zone": the perfect amount of information to reveal.

The Two Types of Humans: The "Detective" vs. The "Reader"

The most brilliant part of this paper is how they categorize how humans process information. They identify two types of people:

1. The Detective (The Sophisticated Agent): This person doesn't just look at the data; they look at why the AI chose that data. If the AI highlights "High Blood Pressure," the Detective thinks, "Wait, why did the AI pick that? Is it because the blood pressure is high, or because the AI knows that blood pressure is the most important thing for this specific patient?" They read between the lines.
2. The Reader (The Naive Agent): This person is more straightforward. They see "High Blood Pressure" and think, "Okay, blood pressure is high." They don't wonder about the AI's motives; they just take the facts at face value.

The "Double-Edged Sword" of Intelligence

The researchers discovered a massive conflict between these two types of people.

If you design an AI specifically for the Detective, it might try to be "clever." It might use a complex pattern of highlighting to signal hidden information. But if you give that "clever" AI to a Reader, they will be totally confused. They won't see the hidden signal, and the AI's "cleverness" might actually lead them to a wrong conclusion. This is what they call the "Price of Complexity."

Conversely, if you design an AI to be super simple for the Reader, the Detective might find it boring and miss out on the deeper insights the AI was trying to signal. This is the "Price of Simplicity."

The Solution: The "Surprise" Strategy

So, what works best? The researchers found that instead of always showing the same "important" things (like always showing a person's GPA in a job application), the AI should show "Surprises."

Think of it like a weather app. If it’s a sunny day in California, the app doesn't need to tell you "It is sunny." That’s not news. But if it’s unexpectedly snowing in Los Angeles, the app should scream that from the rooftops!

The paper shows that an algorithm that highlights "contextual surprises"—the things that are most unexpected for that specific case—is incredibly effective. They tested this using real estate data (the American Housing Survey) and found that by highlighting only the most "surprising" features of a house, they could help people estimate its value much more accurately than if they just showed a standard list of features.

The Takeaway

The paper concludes that we shouldn't try to build "perfectly smart" AIs that try to outsmart human detectives. Instead, we should build smart, simple, and robust tools.

The best AI assistant isn't the one that tries to be a genius; it's the one that knows exactly which "surprises" to put on your sticky note so you can do your job better.

Technical Summary

Technical Summary: Algorithmic Feature Highlighting for Human–AI Decision-Making

Authors: Yifan Guo and Jann Spiess (Stanford University)
Date: April 23, 2026

---

1. Problem Statement

In high-stakes human–AI collaboration (e.g., medical diagnosis, hiring, or real estate appraisal), algorithms often assist rather than replace humans. A critical bottleneck is the human's limited cognitive bandwidth: they cannot inspect all available features of a complex case.

The authors address the problem of algorithmic feature highlighting: instead of providing a single prediction (which may be uninterpretable or untrustworthy), the algorithm selects a small subset of existing, verifiable features to reveal to the human. The central challenge is determining the optimal information policy—which features to highlight—given that the human's effectiveness depends heavily on how they interpret the algorithm's selection.

2. Methodology

The paper models the interaction as a principal–agent game where the algorithm designer (principal) chooses a highlighting policy $\sigma$ to minimize expected loss, and the human (agent) makes a decision based on the revealed features.

A. Human Decision-Making Models:
The authors distinguish between two fundamental modes of belief updating:

• Sophisticated Agent: A Bayesian agent who understands the algorithm's selection rule. They condition their posterior belief not only on the values of the revealed features but also on the fact that those specific features were chosen (the selection event).
• Naive Agent: An agent who ignores the selection rule. They update their beliefs based solely on the revealed feature values, treating the selection as exogenous/random.

B. Optimization Framework:
The goal is to find a policy $\sigma$ that minimizes the expected loss $R_\tau(\sigma)$ for an agent of type $\tau \in \{\text{Sophisticated, Naive}\}$. The authors analyze this under various loss functions, primarily squared recovery loss (reconstructing the full feature vector).

C. Algorithmic Approaches:
The paper evaluates several policy types:

• Fixed Subset Selection: Highlighting the same features for every case (ex-ante).
• Contextual Highlighting: Selecting features based on the specific instance (ex-post).
• Greedy Information Gain: A sequential approach that selects features one by one to maximize the reduction in realized loss.
• Deviation Heuristic: Highlighting features that deviate most from their marginal means (a "surprise-based" approach).

3. Key Contributions and Results

A. The "Price of Complexity" and "Price of Simplicity":
The authors prove that optimizing for one type of agent can be disastrous for the other:

• Price of Complexity: A policy optimized for a sophisticated agent can perform arbitrarily poorly for a naive agent because the sophisticated policy relies on the "information in the selection," which misleads the naive agent.
• Price of Simplicity: A policy optimized for a naive agent can be arbitrarily suboptimal for a sophisticated agent, as it fails to exploit the rich informational signals embedded in the selection rule.

B. Computational Complexity:

• Sophisticated Updating is NP-Hard: Even with binary features and a bandwidth of $k=1$, finding the optimal policy for a sophisticated agent is NP-hard (via reduction from $k$-means clustering).
• Naive Updating is Tractable: For a fixed bandwidth $k$, the optimal policy for a naive agent can be computed in polynomial time via enumeration.

C. Asymptotic and Structural Results:

• For independent Bernoulli features, the authors provide closed-form expressions for the asymptotic risk.
• They show that contextual highlighting strictly dominates fixed subset selection.
• They demonstrate non-monotonicity: in correlated settings, revealing more information can actually increase loss for a naive agent (as it can introduce noise into the posterior).

D. Empirical Validation (American Housing Survey):
Using a calibrated model of 1,521 homes and 44 housing characteristics, the authors show:

• Contextual greedy algorithms significantly outperform fixed policies (e.g., cutting loss by more than half at $k=10$).
• A "Smart Greedy" algorithm (with early stopping) can outperform even a "full reveal" of all features by withholding noisy, low-value information.

4. Significance

This paper shifts the focus of Human-AI interaction from what the AI predicts to how the AI surfaces information.

1. Practicality: It advocates for contextual, greedy, and robust highlighting policies. Since sophisticated optimization is computationally intractable and fragile, the authors suggest that designing for "naive" or "partially sophisticated" agents is a more implementable and robust engineering goal.
2. Interpretability: By highlighting existing features rather than generating LLM-based summaries, the framework ensures the information provided is verifiable and trustworthy.
3. Theoretical Insight: It provides a rigorous mathematical foundation for understanding the "information in the selection," proving that the act of choosing what to show is itself a powerful (and potentially dangerous) signal.