🤖 AI

Results-Actionability Gap: Understanding How Practitioners Evaluate LLM Products in the Wild

Through interviews with nineteen practitioners, this paper identifies the "results-actionability gap" as a key challenge in evaluating LLM-powered products and proposes strategies to bridge the divide between informal interpretive practices and systematic evaluation frameworks.

Original authors: Willem van der Maden, Malak Sadek, Ziang Xiao, Aske Mottelson, Q. Vera Liao, Jichen Zhu

Published 2026-04-21

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Willem van der Maden, Malak Sadek, Ziang Xiao, Aske Mottelson, Q. Vera Liao, Jichen Zhu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a chef who just bought a new, super-smart robot kitchen assistant. This robot can cook almost anything, but it's a bit unpredictable. Sometimes it makes a masterpiece, and other times it accidentally puts salt in your dessert instead of sugar.

Your job is to make sure this robot is safe and delicious before you serve food to your customers. But here's the problem: You don't have a standard recipe book for testing this robot.

This is exactly the situation described in the paper "Results-Actionability Gap: Understanding How Practitioners Evaluate LLM Products in the Wild." The researchers talked to 19 people (chefs, engineers, designers) who are trying to build products using Large Language Models (LLMs)—the technology behind AI chatbots—and they discovered some fascinating truths about how these teams are struggling and succeeding.

Here is the story of their findings, broken down into simple concepts.

1. The Old Rules Don't Work

In the past, if you built a calculator app, you could test it perfectly. If you typed 2 + 2, it had to say 4. If it said 5, you knew exactly what was broken.

But LLMs are like improvisational jazz musicians. They don't follow strict rules; they guess the next note based on probability.

The Problem: You can't just run a checklist. If the AI writes a funny joke, how do you measure "funny"? Is it 8 out of 10? Is it 9?
The Reality: The researchers found that teams are trying to use old, rigid testing tools (like standardized math tests) on these jazz musicians, and it's not working. The tests are "bordering on useless."

2. The "Vibe Check" is Actually Genius

Because the old tests fail, teams have started doing something very human: The Vibe Check.

Imagine you are tasting a new soup. You don't pull out a chemical analyzer immediately. You take a spoon, taste it, and think, "Hmm, this feels a bit too salty, but the texture is nice."

What the paper calls it: "Vibe checks" or "gut feeling" evaluations.
What the researchers found: They used to think this was unprofessional or lazy. But they realized it's actually necessary. Because the AI is so unpredictable, you need a human to use their intuition to spot weird, context-specific errors that a computer metric would miss.
The Shift: Instead of trying to replace the "vibe check" with a robot, we should learn how to make the "vibe check" more organized.

3. The Big Problem: The "Results-Actionability Gap"

This is the most important discovery in the paper. It's the title's main concept.

Imagine you are the head chef. You taste the soup (the evaluation) and say, "This tastes weird. It's a 4 out of 10."
You hand that score to your team of engineers (the prompt engineers, the database managers, the model trainers).
They ask: "Okay, but what do we change to fix it?"

Did we use too much salt (the prompt)?
Did the robot grab the wrong ingredients from the fridge (the data retrieval)?
Is the robot just having a bad day (the model itself)?

The Gap: You have the result (the bad score), but you have no idea how to fix it.
The paper calls this the Results-Actionability Gap. Teams are collecting data, but they can't translate that data into a specific action. It's like having a car engine light on, but the mechanic says, "Well, it's definitely the engine, but we don't know which part, so we'll just keep driving and hope it doesn't explode."

4. Why Is This Happening? (The Four Culprits)

The researchers identified four reasons why fixing these AI products is so hard:

The Generalist vs. Specialist Mismatch: The AI is a "generalist" (it knows a little about everything). Your product is a "specialist" (it needs to be perfect at one specific thing, like legal advice). The AI drifts off-topic easily.
No "Right Answer": In math, there is one right answer. In AI, "good" is subjective. One person thinks a joke is hilarious; another thinks it's offensive. There is no ground truth to measure against.
The Black Box: When the AI fails, it's often because of a messy combination of five different settings. Changing one thing might break something else. It's like trying to fix a Swiss Army knife by only looking at the scissors.
Too Many New Tools: There are so many new ways to test AI that teams get overwhelmed and just stick to messy spreadsheets, which creates "technical debt" (a mess that gets harder to clean up later).

5. How Do Successful Teams Fix It?

The paper doesn't just point out problems; it offers three "recipes" for success based on teams that are doing it right:

Recipe 1: Bake Evaluation into the Design (Evaluation-by-Design)
Don't wait until the cake is baked to taste it. Start tasting while you are mixing the batter. Decide before you build the product what "success" looks like and how you will check it.
Recipe 2: Keep a "Failure Log" (Continuous Sense-Making)
Don't just say "this was bad." Write down why it was bad. If the AI sounded rude, note exactly what prompt caused it. Turn your "vibes" into a shared notebook so the whole team learns from mistakes.
Recipe 3: Change One Thing at a Time (Incremental Testing)
Don't throw the whole recipe away and start over. If the soup is salty, change only the salt. If it's still bad, change only the pepper. This helps you trace exactly what caused the problem.

The Bottom Line

The paper tells us that the people building AI products aren't failing because they are bad at their jobs. They are struggling because the technology is fundamentally different from what we are used to.

We can't force AI to be a calculator. We have to accept that it's a jazz musician. Instead of trying to force it into a rigid box, we need to support the humans who are using their intuition ("vibe checks") to guide it, and help them turn those gut feelings into clear, actionable steps.

In short: Stop trying to measure the unmeasurable with a ruler. Start teaching teams how to listen to the music, write down the notes, and tweak the song one instrument at a time.

1. Problem Statement

As organizations integrate Large Language Models (LLMs) into production systems, traditional evaluation frameworks (benchmarks, automated metrics) have proven inadequate. The probabilistic, non-deterministic, and context-dependent nature of LLMs creates a "socio-technical gap" where technical metrics fail to capture user value or safety in real-world deployments.

While academic research offers numerous frameworks, there is a lack of empirical understanding regarding how practitioners (engineers, designers, product managers) actually evaluate LLM products in diverse, resource-constrained production environments. A critical, previously untheorized problem exists: even when teams successfully collect evaluation data, they often cannot translate these findings into concrete system improvements. This paper seeks to identify current evaluation practices, uncover the specific challenges practitioners face, and propose strategies to bridge the gap between data collection and actionable system refinement.

2. Methodology

Study Design: Qualitative, semi-structured interviews.
Participants: $N=19$ $N = 19$ practitioners from diverse sectors (healthcare, legal, education, enterprise software, consumer apps) and organizational sizes (startups to Fortune 500).
- Roles: Data scientists, software engineers, designers, researchers, and marketing professionals.
- Context: Participants primarily work with foundation models via APIs (prompt engineering, RAG, fine-tuning small local models) rather than training base models from scratch. They evaluate complete systems (UI + retrieval + model), not just isolated model capabilities.
Data Collection: Conducted remotely (Feb–May 2025), lasting 45–60 minutes.
Analysis: Reflexive thematic analysis (Braun & Clarke). The team developed an inductive codebook through iterative coding, consolidating codes like "organizational tradeoffs" and adding new ones like "actionability of results" as patterns emerged.

3. Key Contributions

The paper makes three primary contributions to HCI and LLM evaluation practice:

Empirical Taxonomy of Practices: Identification of 10 distinct evaluation activities spanning execution, design, and organizational meta-work.
Theoretical Conceptualization: Introduction of the "Results-Actionability Gap," a novel challenge where evaluation data is collected but cannot be translated into specific system fixes due to the inability to isolate failure root causes in complex, interacting LLM systems.
Strategic Framework: Proposal of three actionable strategies to bridge the gap, shifting the focus from developing new metrics to supporting the "formalization journey" of practitioners from ad-hoc intuition to systematic evaluation.

4. Key Results

A. Ten Evaluation Practices (RQ1)

The authors categorized practitioner activities into three groups:

Evaluation Execution (How they evaluate):
- A1. Vibe Checks: Informal, formative "gut feeling" tests (e.g., "hammering" the system) used as a first line of defense.
- A2. User Feedback: Continuous collection of end-user signals (thumbs up/down, think-aloud protocols).
- A3. Expert Collaboration: Ongoing engagement with domain experts (e.g., lawyers, doctors) acting as "living benchmarks."
- A4. Automated Tests: Attempts to use traditional ML metrics (BLEU, F1) or LLM-as-Judge, often found to be "bordering on useless" for specific contexts.
Evaluation Design (How they design approaches):
- A5. Construct Extraction: Translating vague qualitative observations (e.g., "it feels right") into specific technical constructs.
- A6. Pragmatic Metric Selection: Choosing metrics based on actionability and stakeholder communication rather than pure rigor.
- A7. Systematizing Toolkits: Attempts to codify ad-hoc tools into reusable frameworks (e.g., STPA matrices).
Organizational Meta-Work (How they navigate the organization):
- A8. Alignment: Workshops to disambiguate terms and agree on shared evaluation goals.
- A9. Documentation: Creating "metric cards," prompt libraries, and ground-truth datasets to preserve institutional memory.
- A10. Advocacy: Strategic communication to secure resources and prioritize evaluation work.

B. Five Key Challenges (RQ2)

The study confirms four known challenges and introduces a fifth:

C1. Aligning Objectives: Difficulty getting stakeholders (technical, business, user) to agree on what "success" means.
C2. Establishing Constructs: Defining what "good" looks like in the absence of ground truth.
C3. Choosing Approaches: Uncertainty in selecting viable methods due to a lack of tested frameworks.
C4. Technical Barriers: Infrastructure limitations, non-determinism, and scaling human evaluation.
C5. The Results-Actionability Gap (Novel): The inability to trace a negative evaluation result (e.g., a low score or bad "vibe") to a specific, fixable component (prompt, retrieval, temperature, or model). Practitioners gather data but cannot determine which dial to turn to fix the issue.

C. Structural Factors Driving Interpretation

The authors argue that reliance on interpretive methods (like "vibe checks") is not a methodological failure but a necessary adaptation to four structural factors of LLMs:

Mismatch: General-purpose models vs. specific product contexts.
Non-determinism + Absent Ground Truth: Outputs vary, and "correctness" is often perspectival.
Failure of Quantitative Signals: Traditional metrics (accuracy) do not correlate with user experience in open-ended tasks.
Adoption Barriers: The high cost of learning new frameworks leads to technical debt.

5. Significance and Implications

For HCI Researchers

Reframing the Role of HCI: Instead of creating new, rigid evaluation frameworks that practitioners may abandon, HCI should focus on systematizing existing ad-hoc practices.
Supporting the Formalization Journey: Researchers should help practitioners make their tacit knowledge (e.g., "vibes") explicit and shareable, bridging the gap between intuitive heuristics and rigorous methodology.
Methodological Fit: Existing HCI methods (contextual inquiry, participatory design, ethnography) are uniquely suited to address the socio-technical nature of LLM evaluation.

For Practitioners

Validation: Current struggles are inherent to the technology, not a sign of poor practice.
Actionable Strategies: To bridge the Results-Actionability Gap, teams should:
1. Evaluation-by-Design: Integrate evaluation criteria into the initial design process (e.g., using STPA to define failure modes early).
2. Continuous Sense-Making: Maintain lightweight logs converting "vibes" into testable hypotheses and documenting decisions to avoid reinventing the wheel.
3. Incremental Changes: Change one variable at a time (e.g., only the prompt) to isolate root causes, rather than overhauling entire systems.

Conclusion

The paper concludes that the "Results-Actionability Gap" is the primary bottleneck in LLM product development. Because LLM systems involve unpredictable interactions between prompts, retrieval, and models, a low score does not automatically indicate what to fix. The solution lies not in better metrics, but in organizational and process adaptations that allow teams to trace qualitative insights back to specific, actionable system components.