💬 NLP

Decide less, communicate more: On the construct validity of end-to-end fact-checking in medicine

This position paper argues that end-to-end fact-checking systems are ill-suited for medicine due to fundamental challenges in connecting real-world claims to scientific evidence, proposing instead that fact-checking be reimagined as an interactive communication process.

Original authors: Sebastian Joseph, Lily Chen, Barry Wei, Michael Mackert, Iain J. Marshall, Paul Pu Liang, Ramez Kouzy, Byron C. Wallace, Junyi Jessy Li

Published 2026-04-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sebastian Joseph, Lily Chen, Barry Wei, Michael Mackert, Iain J. Marshall, Paul Pu Liang, Ramez Kouzy, Byron C. Wallace, Junyi Jessy Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Why "Fact-Checking" Medicine is Harder Than You Think

Imagine you are trying to build a robot that acts like a super-librarian. Its job is simple: You ask it a question about your health (like, "Does eating grapefruit stop my seizure meds from working?"), and it instantly checks the world's medical books, finds the answer, and gives you a clear "Yes," "No," or "Maybe."

This paper argues that we have been trying to build this robot wrong. We have been treating medical fact-checking like a multiple-choice test where the robot just picks an answer. The authors say this approach is broken because real-life health questions are messy, confusing, and often unanswerable by the strict rules of science.

Instead of a robot that just decides the answer, we need a robot that talks to you like a doctor does.

The Experiment: Asking the Experts to Play the Game

To figure out why the current "robot librarian" approach fails, the authors didn't just write code. They hired five real medical experts (doctors and researchers) to play the role of the robot.

They gave these experts:

Real questions posted by regular people on Reddit (e.g., "Can pineapple juice help my sinus infection?").
A list of 10 scientific studies (Randomized Controlled Trials) that a computer found to be related.
The task: Read the studies, decide if the Reddit question is true or false, and explain why.

The Result? The experts couldn't agree. Even with the same evidence, they gave different answers. This proved that the current way we try to automate fact-checking is fundamentally flawed.

The Three Big Problems (The "Why It Failed" Section)

The paper identifies three main reasons why a simple "Yes/No" robot doesn't work in medicine:

1. The "Missing Puzzle Piece" Problem (Unverifiable Claims)

The Analogy: Imagine you ask a detective, "Did the butler steal the vase?" but the detective has no evidence because the butler was never in the room. The detective can't say "Yes" or "No"; they have to say, "I can't check this."

The Reality: Many health questions people ask online are unverifiable.

Some questions are about things that are unethical to test (e.g., "Does smoking cause cancer?" We can't run a study where we force people to smoke).
Some are too specific (e.g., "Does grapefruit interact with this specific drug for this specific person?"). No study exists for that exact combination.
The Finding: In the study, the experts often had to say, "There is no relevant evidence." A robot that just tries to force a "True/False" label will get this wrong.

2. The "Vague Question" Problem (Underspecified Claims)

The Analogy: Imagine you ask a mechanic, "My car is making a noise." The mechanic can't fix it because they don't know which car, what noise, or when it happens. If the mechanic guesses, they might fix the brakes when the problem is the engine.

The Reality: People on social media ask vague questions.

Example: "Herbs help regulate my cycle." What does "regulate" mean? Does it mean stopping the period? Making it shorter? Fixing cramps?
The experts in the study interpreted this differently. One thought it meant "stopping the period," another thought it meant "making it regular."
The Finding: Because the questions are so vague, the "answer" changes depending on what the person actually meant. A robot that doesn't ask for clarification will give the wrong answer.

3. The "Subjective Truth" Problem (Labeling Severity)

The Analogy: Imagine a teacher grading a student's essay. One teacher thinks a small grammar mistake is a "C." Another teacher thinks it's a "B." Both are right, but they have different standards.

The Reality: Even when experts agree on the facts, they disagree on how bad a mistake is.

Example: Someone says, "Pineapple juice cures sinus infections quickly."
Expert A says: "That's mostly true, just maybe not that fast." (Label: Partially Supports).
Expert B says: "That's dangerous misinformation because it implies a quick fix." (Label: Refutes).
The Finding: There is no single "correct" label. It depends on the expert's philosophy and how much risk they think the patient faces.

The Solution: The "Doctor-Patient" Conversation

Since a robot that just decides is failing, the authors propose a new model: The Interactive Dialogue.

Instead of a robot that says, "Your claim is False," the system should act like a doctor talking to a patient.

How it works:

Ask for Clarification: If the patient says, "Herbs help my cycle," the system asks, "What do you mean by 'regulate'? Do you mean stopping it or making it regular?"
Guide the Search: If the patient asks about something that can't be tested (like an unethical experiment), the system says, "We can't test that directly, but here is what we know about similar situations."
Show Different Views: Instead of forcing one "True/False" label, the system explains, "Some experts think this is a minor issue, while others think it's risky. Here is why they differ."

The Conclusion

The paper concludes that medical fact-checking isn't a math problem; it's a conversation.

Trying to force complex, messy human health questions into a simple "True/False" box is like trying to fit a square peg in a round hole. It doesn't work, and it leads to confusion.

To fix this, we need to stop trying to build systems that just decide the answer and start building systems that communicate with the user to understand what they really need. As the title says: Decide less, communicate more.

1. Problem Statement

The paper addresses the limitations of current end-to-end automated fact-checking systems when applied to the medical domain. While advancements in Large Language Models (LLMs) and retrieval systems have improved fact-checking in general domains, these systems struggle with real-world medical claims found on social media (e.g., Reddit).

Key issues identified include:

Contextual Disconnect: Existing datasets often use curated claims or decontextualized statements, failing to capture the nuance of layperson inquiries.
Underspecification: Real-world claims are often vague, missing critical details (Population, Intervention, Outcome), or based on incorrect premises.
Unverifiability: Many claims cannot be verified by Randomized Controlled Trials (RCTs) due to ethical constraints, lack of feasibility, or the specific nature of the query.
Subjectivity: Veracity labels (True/False) are inherently subjective in medicine, leading to low inter-annotator agreement even among experts.
Construct Validity: The authors argue that framing fact-checking as a static, end-to-end classification task lacks construct validity for complex medical discourse.

2. Methodology

To investigate these challenges, the authors conducted an "AI-in-the-loop" expert annotation study designed to establish an "idealized upper bound" for automated systems.

Data Source: Claims were extracted from the RedHOT corpus (Reddit posts regarding 24 health conditions).
Participants: Five clinical experts (medical professionals) with experience in evidence synthesis.
Pipeline (Figure 1):
1. Claim Extraction: Claims were contextualized using PIO descriptors (Population, Intervention, Outcome) extracted via an LLM pipeline.
2. Evidence Retrieval: An automated dense retrieval system (using stella_en_400M_v5 embeddings) retrieved 10 relevant RCT abstracts from Trialstreamer for each claim.
3. Annotation Task: Experts performed a multi-step verification process:
  - Relevance Assessment: Labeled retrieved abstracts as Relevant, Somewhat Relevant, or Irrelevant across PIO dimensions.
  - Evidence Synthesis: Hierarchically tiered abstracts based on relevance and quality.
  - Veracity Judgment: Provided two judgments:
    - Overall Support: Based strictly on retrieved evidence.
    - Expert Support: Based on clinical expertise (for claims unlikely to be in RCTs).
  - Explanation: Wrote natural language justifications for their labels.
Labels: A 6-class label set was used: No Relevant Abstracts, Refutes, Partially Refutes, Inconclusive, Partially Supports, Supports.

3. Key Findings & Results

The study revealed fundamental obstacles that challenge the feasibility of end-to-end fact-checking:

Low Inter-Annotator Agreement (IAA): Even with shared evidence and expert guidance, agreement was low.
- Cohen's $\kappa$ scores ranged from -0.184 to 0.416.
- Most notably, 20 out of 25 expert judgments in the final round were labeled "No Relevant Abstracts," indicating that the majority of real-world claims are unverifiable via RCTs.
Challenge 1: Unverifiable Claims: Claims often lack an intervention, involve unethical interventions (e.g., smoking), or involve PIO combinations that are practically impossible to study in RCTs.
Challenge 2: Interpretation Variance:
- Underspecification: Laypeople often omit timeframes or specific mechanisms (e.g., "herbs regulate the cycle"), leading to contradictory expert interpretations.
- Misguided Premises: Claims often rest on false assumptions (e.g., ADHD medication efficacy linked to menstrual cycles). Experts disagreed on whether to refute the premise or the conclusion.
Challenge 3: Subjectivity of Severity: Experts often agreed on the reasoning (e.g., "pineapple juice helps but slowly") but disagreed on the label (e.g., is a slow-acting remedy a "False" claim or a "Partially Supports" claim?). This highlights that a single veracity label is insufficient.

4. Proposed Solution: The Communication Model

The authors argue that fact-checking should shift from an end-to-end classification task to an interactive communication model (Figure 2).

Clarifying Intent: Instead of guessing the user's intent, the system should engage in dialogue to resolve underspecifications (e.g., asking "What do you mean by 'regulating' the cycle?").
Guided Retrieval: If no evidence exists, the system should not simply output "False." Instead, it should guide the user toward related, verifiable claims or explain why the specific claim is unverifiable (e.g., "No RCTs exist for this specific drug interaction due to ethical constraints").
Diverse Perspectives: Rather than forcing a consensus label, the system should present diverse expert viewpoints and nuanced explanations, acknowledging the inherent subjectivity of medical truth.
Expanded Evidence: The model should integrate non-RCT evidence (cohort studies, case reports) when RCTs are unavailable, clearly communicating the quality and limitations of such evidence.

5. Significance and Contributions

First Expert Study on "In-the-Wild" Medical Claims: This is the first study to examine how clinical experts verify real social media claims using synthesized evidence, providing a benchmark for the upper bound of current AI capabilities.
Construct Validity Critique: The paper provides empirical evidence that the standard "Claim $\to$ Evidence $\to$ Label" pipeline is flawed for medicine. It demonstrates that high-stakes medical decisions require dialogue, not just classification.
New Dataset & Guidelines: The authors release a dataset of 1,000 abstract-level annotations and 100 synthesis-level explanations, along with detailed guidelines for medical fact-checking.
Paradigm Shift: The paper advocates for Human-in-the-Loop (HITL) and Human-AI collaboration where the AI acts as a conversational agent that clarifies intent and manages uncertainty, rather than a deterministic fact-checker.

Conclusion

The paper concludes that "deciding less" (avoiding premature, rigid classification) and "communicating more" (engaging in clarifying dialogue) is the necessary path forward for medical fact-checking. This approach better aligns with the realities of clinical practice, where physicians constantly clarify patient intent and navigate uncertainty, rather than simply assigning binary labels to health claims.