🤖 AI

Epistemic Blinding: An Inference-Time Protocol for Auditing Prior Contamination in LLM-Assisted Analysis

This paper introduces "epistemic blinding," an inference-time protocol that anonymizes entity identifiers to audit and quantify the extent to which large language models rely on memorized priors versus supplied data in agentic analysis, revealing significant contamination in drug target prioritization and financial screening while ensuring the system's adherence to designed analytical processes.

Original authors: Michael Cuccarese

Published 2026-04-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Michael Cuccarese

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a brilliant, well-read consultant to help you pick the best candidates for a job. You give them a stack of resumes with specific data: years of experience, test scores, and project outcomes. You ask them to rank the top 20 candidates based only on that data.

However, this consultant has a secret habit. Before you even handed them the resumes, they had read a famous biography about one of the candidates, let's call him "John." They know John is famous, charismatic, and has a great reputation.

When you ask the consultant to rank the list, they might put John at #1. But here's the catch: they didn't put him there because his test scores were the highest. They put him there because their brain automatically filled in the gaps with what they already knew about John. They couldn't tell you how much of their decision came from the resume you gave them, and how much came from their memory of the famous biography.

This is the problem this paper solves.

The paper introduces a simple trick called "Epistemic Blinding." It's a way to audit whether your AI (the consultant) is actually looking at the data you gave it, or if it's just relying on its own "memories" (training data).

Here is the breakdown using simple analogies:

1. The Problem: The "Famous Name" Bias

Large Language Models (LLMs) are like super-readers. They have read almost everything on the internet. When you ask them to analyze data about a specific gene (in biology) or a specific company (in finance), they instantly recognize the name.

The Issue: If the name is famous (like the gene KRAS or the company Apple), the model's brain lights up with everything it knows about them. It might say, "This is a great target because it's a famous drug target!"
The Danger: That "famous drug target" fact didn't come from the data you provided. It came from the model's training. The model silently blends the two, and you have no way of knowing if it's making a smart decision based on your numbers or just repeating what it learned in school.

2. The Solution: The "Anonymous Code" Trick

The authors propose a protocol called Epistemic Blinding. Think of it like a double-blind clinical trial, but for AI.

Instead of letting the AI see the names "KRAS" or "Apple," you replace them with anonymous codes like Gene_088 or Company_42.

Step 1: You take your data and swap all the names for random codes.
Step 2: You ask the AI to rank the list based only on the numbers (scores, frequencies, etc.) attached to those codes.
Step 3: You do the exact same thing again, but this time you let the AI see the real names.
Step 4: You compare the two lists.

3. What Happens When You Compare?

The paper ran this experiment in two very different worlds: Cancer Research and Stock Market Investing.

In Cancer Research: When the AI couldn't see the gene names, it ranked a gene called Gene_088 (which is actually KRAS) at #5. But when it could see the name "KRAS," it jumped to #1.
- Why? The AI knew KRAS was famous and "druggable" from its training. It ignored the fact that other genes in the data had stronger statistical signals.
- Result: Blinding changed 16% of the top 20 predictions. It didn't change the valid targets (the ones that actually work), but it changed which new candidates the AI thought were promising.
In Stock Market Investing: The AI was asked to find the best "value" stocks. When it saw the ticker symbols (like ELV or CI), it promoted famous companies. When they were anonymous codes, it picked different companies based strictly on the math.
- Result: 35% of the top 20 recommendations changed just by hiding the names.

4. Why This Matters (The "Audit" Analogy)

The author isn't saying that the AI is "wrong" when it uses its memory. Sometimes, knowing that KRAS is a famous drug target is actually helpful!

But here is the critical point:
If you don't use blinding, you are flying blind. You don't know if the AI is acting as a scientist (analyzing your data) or a student (reciting what it memorized).

Without Blinding: The AI is a black box. You get an answer, but you can't trace why.
With Blinding: You get a "truth meter." If the AI gives you a totally different answer when the names are hidden, you know it was heavily influenced by its own biases, not your data.

5. The Takeaway

The paper argues that whenever you use an AI to make decisions based on data (ranking genes, picking stocks, hiring people), you should use Epistemic Blinding.

It's like a spot check for your AI's brain. It doesn't make the AI smarter, but it makes the AI honest. It forces the AI to prove that it is looking at the evidence you provided, rather than just guessing based on what it already knows.

In short: If you wouldn't let a human judge see the names of the candidates before grading their test scores, you shouldn't let an AI see the names of the data points either. This tool makes that possible with a single click.

1. Problem Statement: Parametric Knowledge Contamination

The paper addresses a critical failure mode in Large Language Model (LLM) assisted analysis: parametric knowledge contamination.

The Issue: When LLMs reason over datasets containing named entities (e.g., gene symbols, stock tickers), they silently blend data-driven inference with memorized priors from their training corpus.
The Consequence: The model's output becomes a "black box" where it is impossible to distinguish how much of a conclusion was derived from the supplied data versus the model's internal training memory.
Bias Mechanism: Entities with high visibility in the training data (e.g., KRAS in oncology, Apple in finance) are systematically promoted, while obscure entities with strong quantitative signals are demoted. This "fame bias" distorts resource allocation and scientific discovery, potentially masking novel candidates.
Current Limitations: Existing solutions like data contamination checks focus on benchmark evaluations, not inference-time reasoning. Anonymization in multi-agent debates usually targets agent identities, not the objects of reasoning.

2. Methodology: Epistemic Blinding Protocol

The authors propose Epistemic Blinding, an inference-time protocol analogous to blinding in clinical trials. It requires no model retraining or architectural changes.

The Protocol Steps:

Entity Identification: Identify columns containing identifiers with uneven training representation (e.g., gene names, tickers).
Mapping & Replacement: Generate a deterministic mapping where every distinct entity is assigned an anonymous code (e.g., Gene_088, Company_042). The same entity receives the same code across all datasets to preserve cross-dataset reasoning.
Leak Mitigation: Identify and neutralize "subtle leak sources" where correlated features (e.g., market cap + sector) could reveal identity. This involves normalizing, binning, or dropping such features.
Shuffling: Randomize row order to remove positional cues.
A/B Comparison:
- Blinded Condition: The LLM receives the anonymized data and is asked to perform the task (e.g., rank top targets).
- Unblinded Control: The LLM receives the original data with real names in a separate session.
De-anonymization & Analysis: Map codes back to real names and compare outputs using metrics like set overlap, rank shifts (Kendall $\tau$ ), and qualitative justification analysis.

Implementation:

Released as an open-source, config-driven tool (blind.py, deblind.py, compare.py).
Implemented as a Claude Code skill for one-command integration into agentic workflows.

3. Key Contributions

Formal Protocol: A rigorous definition of epistemic blinding, including handling cross-dataset consistency and subtle feature leaks.
Open-Source Tooling: A model-agnostic, config-driven implementation that works with any LLM.
Integrated System: A complete oncology target identification system where blinding is applied to both:
- Evolutionary Optimization: LLM-guided evolution of scoring functions operates purely on numerical feature vectors, never seeing gene names.
- Agentic Reasoning: LLM rationalization of top candidates is performed under blinded conditions.
Empirical Validation: Controlled experiments demonstrating systematic fame bias in both biological and financial domains.
Domain Generalization: Proof that the contamination problem and the blinding solution apply beyond biology to fields like equity screening.

4. Experimental Results

A. Oncology Drug Target Prioritization

Setup: An LLM ranked top 20 drug targets across four cancer types (AML, Pancreatic, CRC, GBM) using 8 data layers (mutations, embeddings, etc.).
Findings:
- Rank Shift: Blinding changed 16% of the top-20 predictions on average.
- Target Recovery: Both blinded and unblinded models recovered the same number of validated FDA-approved targets (e.g., 2.75 per indication), proving blinding does not degrade the recovery of known biology.
- Fame Bias:
  - KRAS (Colorectal Cancer): Ranked #1 when named (justified by "proven therapeutic tractability"), but dropped to #5 when blinded (ranked by mutation frequency/convergence).
  - DPP8 (Glioblastoma): Ranked #3 when blinded (strongest convergence signal), but dropped to #9 when named, displaced by famous genes like RB1 and PIK3R1.
- Mechanism: Justifications in the unblinded condition frequently cited external knowledge (e.g., "covalent RAS inhibitors") not present in the provided data.

B. S&P 500 Equity Screening

Setup: LLM ranked top 20 value investments based on fundamentals (P/E, growth, margins).
Findings:
- Rank Shift: Revealing tickers reshaped 30–40% of the top-20 rankings across five random seeds.
- Systematic Bias: Famous large-cap stocks (e.g., ELV, CI) were promoted when named, while obscure but fundamentally strong companies were demoted.
- Conclusion: The contamination mechanism is domain-agnostic and operates identically in finance.

5. Significance and Implications

Auditability over Accuracy: The paper argues that the primary value of epistemic blinding is auditability, not necessarily improved accuracy. It restores a critical axis of traceability: determining whether an agent is reasoning from provided data or memorized priors.
Paradigm Shift: It challenges the assumption that LLMs are neutral data processors. It demonstrates that "data-driven" analysis with LLMs is often a hybrid of data and training memory.
Practical Adoption: By embedding the protocol as a seamless tool (Claude Code skill), the authors lower the barrier to adoption, making blinding a default step rather than an afterthought.
Limitations: The study acknowledges that blinding does not fix sampling bias in the data itself, nor does it eliminate structural leakage (where features uniquely identify an entity). It also notes that for tasks requiring synthesis of known knowledge (e.g., literature review), blinding is counterproductive.

Conclusion: Epistemic blinding is a necessary protocol for any LLM-assisted analysis involving named entities. Without it, researchers cannot verify if their agents are adhering to the analytical process designed or if they are merely regurgitating training biases.