📊 statistics

Prior laundering: learned priors with inherited, undetectable overconfidence

This paper warns that using legacy reconstructions as training data for generative priors—a practice termed "prior laundering"—leads to undetectable overconfidence and poor uncertainty quantification in ill-posed inverse problems because the model inherits and amplifies the archive's biases rather than learning from actual data.

Original authors: Ali Siahkoohi, Sina Alemohammad

Published 2026-07-27

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Ali Siahkoohi, Sina Alemohammad

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Mystery of the Invisible Map

Imagine you are a detective trying to solve a crime, but you only have a blurry, incomplete photo of the scene. You know the laws of physics (how light bends, how sound travels), but the photo is missing huge chunks of information. To fill in the gaps, you need a "prior"—a set of educated guesses about what the missing parts usually look like. In the world of science, this is called an inverse problem: working backward from a messy result to figure out the original cause.

For decades, scientists used hand-crafted rules for these guesses, like "assume the image is smooth" or "assume the object is simple." But recently, they started using AI to learn these rules. The AI looks at thousands of past solutions and learns what a "plausible" answer looks like, creating a generative prior. This is a powerful tool, but it has a catch: to train the AI, you need to know the true answer (the "ground truth"). In fields like seismology (looking inside the Earth) or medical imaging (looking inside the human body), the true answer is almost never available. You can't see the exact rock layers underground or the exact healthy tissue inside a patient without invasive surgery.

So, what do scientists do? They cheat a little, or so they think. They take the old guesses made by previous methods, treat them as if they were the truth, and train their new AI on them. This paper calls this process "prior laundering." It's like taking a sketch drawn by a previous artist, washing it in a machine to make it look like a photograph, and then claiming the new photo is based on real data. The big question is: Does this trick work, or does it hide a dangerous secret?

The Laundry Cycle: How a "Clean" AI Gets Dirty

This paper, titled "Prior Laundering: learned priors with inherited, undetectable overconfidence," investigates exactly what happens when you train a powerful AI on these "legacy" reconstructions instead of real truths. The authors, Ali Siahkoohi and Sina Alemohammad, discovered a troubling phenomenon: the AI doesn't just learn from the data; it inherits the blind spots and false confidence of the old method it was trained on.

Here is the core of the problem, explained through a simple analogy:

Imagine you are trying to guess the contents of a sealed box. You can shake it and hear some rattling (the measurements).

The Resolved Directions: If the box has a loose coin inside, you can hear it clearly. The data tells you exactly where the coin is.
The Blind Directions: But the box also has a heavy, silent block of lead that doesn't rattle at all. No matter how hard you shake the box, the data tells you nothing about the lead block.

In a perfect world, when the data is silent, you should say, "I have no idea what the lead block looks like; it could be anything." But here is where the "laundering" happens.

The scientists trained their AI on an archive of old guesses. In those old guesses, the people who made them used a rule (a regularizer) that assumed the silent lead block was always a perfect, smooth cube. Because the data couldn't contradict this (since the lead was silent), the old guesses always showed a smooth cube.

When the new AI was trained on these old guesses, it learned a very specific lesson: "When the data is silent, the answer is a smooth cube." The AI didn't learn this from the shaking box (the data); it learned it from the old sketch (the archive).

The Dangerous Illusion of Certainty

The paper finds that this leads to undetectable overconfidence.

When the AI is deployed to solve a new problem, it looks at the silent parts of the data and confidently reports, "I am 99% sure the lead block is a smooth cube!" It draws a very narrow, tight circle around its guess. In statistics, this is called a credible interval. If the interval is too narrow, the AI is "overconfident."

The scary part is that no one can tell the AI is lying.

The Data Check Fails: If you check the AI's answer against the shaking box, it passes. The AI's guess fits the sound perfectly because the sound doesn't care about the lead block.
The Self-Check Fails: Scientists often use a test called simulation-based calibration to see if their AI is honest. They ask the AI to guess, then check if the truth falls inside the AI's confidence circle. But because the AI was trained on the old "smooth cube" rule, it is perfectly consistent with its own training. It passes the test every time, even though it is confidently wrong about the silent parts.

The paper proves mathematically that on these "blind directions," the AI's uncertainty is frozen. It is exactly the same as the old, hand-crafted rule it was trained on. If the old rule was too narrow (too confident), the new AI is too narrow. If the old rule was wrong, the new AI is wrong. The data cannot fix it because the data never "saw" those parts in the first place.

The "Single-Best" Trap

The situation gets even worse with a common practice called single-best curation. Often, archives don't store a range of possibilities (a "cloud" of guesses); they store just one best guess (the "Maximum A Posteriori" or MAP estimate).

The paper shows that if you train an AI on these single, perfect-looking images, the AI's confidence on the blind spots collapses to zero width. It becomes absolutely certain. It says, "I am 100% sure the lead block is a smooth cube."
In reality, the lead block could be a sphere, a pyramid, or a jagged rock. But because the archive only ever showed a cube, the AI thinks the cube is the only possibility. The paper demonstrates that this leads to zero coverage: if you run a test, the true answer will almost never fall inside the AI's tiny, confident circle.

What the Experiments Showed

To prove this wasn't just math on paper, the authors ran simulations on two real-world scenarios:

Seismic Imaging: Looking at underground rock layers. They trained an AI on old "migrated" images. The result? The AI was incredibly confident about the deep, dark parts of the image where the light (data) was weak. It drew tight, narrow bands of uncertainty that were far too small. The "truth" (which they constructed for the test) often fell outside these bands.
Groundwater Flow: Mapping water underground. They used a different type of AI (a normalizing flow) trained on old "best guess" maps. Again, the AI was overconfident in the blind spots, reporting zero uncertainty where there should have been huge uncertainty.

In both cases, the AI trained on the "truth" (the control group) was much more honest. It admitted when it didn't know, showing wide, fuzzy bands of uncertainty. The "laundered" AI, however, looked sharper and more confident, but it was a hallucination of certainty.

The Takeaway: Don't Trust the Cleanest Image

The paper concludes that prior laundering creates a structural flaw that is invisible to standard checks. The AI inherits the "blind faith" of the old method.

The authors suggest a simple, practical fix: Report what the data actually sees.
Before trusting an AI's uncertainty, scientists should calculate which parts of the image the data can actually resolve (the "resolved subspace") and which parts are blind. They should report the confidence for the blind parts as "inherited from the pipeline" rather than "supported by data."

If you are looking at a map of the underground or a scan of a patient, and the AI says, "I am 100% sure about this dark, blurry corner," you should be suspicious. That confidence didn't come from the data; it came from the laundry. The data was silent, and the AI just repeated the old guess with a new, shiny coat of paint.

The paper doesn't say we should stop using AI. It says we must stop pretending that training on old guesses makes the AI "data-driven" in those silent corners. The data has a limit, and when the data stops talking, the AI's confidence is just a ghost of the past.

Technical Summary: Prior Laundering in Learned Generative Priors

Problem Statement
Ill-posed Bayesian inverse problems require a prior to regularize solutions. While learned generative priors (deep generative models) offer sharper reconstructions than handcrafted regularizers, training them requires ground-truth examples, which are scarce in fields like seismic and medical imaging. Consequently, practitioners often train these models on an archive of "legacy reconstructions"—outputs from older, handcrafted methods (e.g., migrated sections or MAP estimates)—rather than true data. This practice, termed "prior laundering," treats the archive as if it were ground truth. The paper investigates the structural consequences of this practice, specifically focusing on the blind subspace: the directions in the solution space that the forward operator cannot resolve (the kernel of the operator). The central question is whether the uncertainty reported by such a prior is truly data-driven or merely an inherited, undetectable artifact of the legacy pipeline.

Methodology and Theoretical Framework
The authors establish a theoretical framework for inverse problems where the unknown $x$ is observed through a noisy forward operator $F(x)$ . They distinguish between the resolved subspace (informative to data) and the blind subspace (uninformative to data).

The "One EM Step" Identity: The paper proves that training a prior on a posterior-sample archive (legacy reconstructions) is mathematically equivalent to advancing the original handcrafted regularizer by exactly one Expectation-Maximization (EM) step toward the truth.
- Resolved Directions: The prior improves where the data provides information.
- Blind Directions: The prior remains "frozen" at the original regularizer's assumption. The learned prior carries the legacy belief into deployment without correction.
Single-Best Curation Collapse: In practice, archives often store single-best (Maximum A Posteriori, MAP) reconstructions rather than posterior samples. The authors prove that training on a single-best archive causes the blind-fiber uncertainty to collapse to zero width. The learned prior reports absolute certainty on directions where the truth genuinely varies, as the deterministic MAP map discards all posterior spread.
Undetectability of Overconfidence: A critical theoretical contribution is the proof that this overconfidence is structurally undetectable using standard deployment checks:
- Data-Driven Checks: Two truths differing only on the blind subspace induce identical measurement laws. Therefore, no statistic computed from the recorded data can distinguish between a correct prior and an overconfident inherited one.
- Self-Consistency Checks: Simulation-based calibration (SBC) is shown to be neutral by construction; it validates the consistency between a prior and its own posterior, not the fidelity to the truth. Thus, SBC passes regardless of the inherited overconfidence.
Coverage Shortfall: In the linear-Gaussian case, the authors derive a closed-form expression for the coverage shortfall. If the inherited belief (the regularizer) is tighter than the truth on the blind subspace, the reported credible intervals are overconfident. This shortfall persists regardless of the amount of data collected, provided the operator's kernel remains non-trivial.

Key Contributions

Characterization of Prior Laundering: The paper formally defines and analyzes the phenomenon where legacy assumptions are re-emitted with the credibility of data-driven uncertainty.
Theoretical Bounds on Undetectability: It establishes that the blind subspace is the only regime where standard deployment diagnostics (data fits and SBC) fail to reveal miscalibration.
Single-Best Collapse: It identifies the specific mechanism by which using single-best archives (common in practice) leads to zero-width credible intervals on blind directions, a more severe failure than simple overconfidence.
Remedies and Limits: The paper proves that the only way to "de-freeze" the blind subspace is to introduce new measurement channels (e.g., additional surveys or different sensor types) that resolve those specific directions. Retraining on fresh data from the same operator only corrects the resolved subspace, leaving the blind conditional frozen at the legacy assumption.

Experimental Results
The authors validate these theoretical claims using two physical operators:

Seismic Born Imaging: A diffusion model trained on a legacy least-squares migration archive.
- Result: The curated prior's uncertainty on the blind subspace (low-illumination regions) was significantly tighter than the truth, leading to under-coverage. Conversely, the uncertainty widened where illumination failed in a truth-trained "oracle" prior. The gradients of uncertainty were opposite: the curated prior was most confident where the data was least informative.
Groundwater Flow Imaging: A normalizing flow trained on legacy MAP estimates.
- Result: Even with a correctly specified regularizer (where the legacy MAP was derived from the true prior), the curated flow under-covered the blind subspace. This confirms that the failure is due to the single-best collapse mechanism (discarding spread) rather than model misspecification.

Significance and Claims
The paper argues that the community's reliance on legacy archives for training generative priors creates a structural blind spot in uncertainty quantification. The significance lies in:

Exposing a Hidden Failure Mode: It demonstrates that "earned" uncertainty in blind directions is actually "inherited" and potentially dangerously overconfident.
Redefining Deployment Standards: It asserts that practitioners must explicitly report which directions are resolved by the operator and which are blind. Confidence intervals on blind directions should be flagged as prior-supplied, not data-driven.
Limiting Expectations: The paper modestly claims that no amount of data from the same survey can fix this issue; the solution requires architectural changes in data acquisition (adding channels) or a shift in how archives are curated (storing posterior samples rather than single-point estimates).

The work concludes that while learned priors improve reconstruction quality, their uncertainty quantification on blind subspaces is fundamentally compromised by the "prior laundering" process, necessitating a separation of data-supported confidence from inherited belief.