🤖 AI

Structural Enforcement of Statistical Rigor in AI-Driven Discovery: A Functional Architecture

This paper presents a formally verified, two-tiered functional architecture that combines a Haskell-based research monad with an OS-level sandbox to structurally enforce statistical rigor and prevent spurious discoveries in AI-driven scientific exploration by guaranteeing online false-discovery-rate control and physically isolating validation data.

Original authors: Karen Sargsyan

Published 2026-07-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Karen Sargsyan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a brilliant, hyper-fast research assistant named "AI-Scientist." This assistant is given a massive task: to test thousands of new ideas overnight to see which ones actually work. It's like a chef trying to invent 2,000 new recipes in one night.

The problem? If you taste-test 2,000 recipes, you will inevitably find about 100 that seem delicious just by pure luck, even if they are actually terrible. In science, this is called a "false discovery." If the AI reports these lucky accidents as real breakthroughs, it creates a mountain of fake news.

This paper proposes a "Safety Fortress" to stop the AI from lying to itself or us. It builds a two-layered defense system, backed by mathematical proof, to ensure that when the AI says "I found something," it really means it.

Here is how the fortress works, using simple analogies:

1. The Problem: The "Budget" of Truth

Think of scientific truth like a limited budget of money.

The Naive Approach: If you let the AI test every idea with a "spending limit" of $0.05 (a 5% chance of being wrong), and it tests 2,000 ideas, it will "spend" that budget 2,000 times. It will run out of money immediately and start printing fake receipts. The paper found that without safeguards, 41% of the AI's "discoveries" were just lucky guesses.
The Solution (LORD++): The paper uses a smarter system called LORD++. Imagine the AI has a single piggy bank of truth-money. Every time it wants to test an idea, it must pay a small fee from the piggy bank.
- If the idea turns out to be a real discovery, the piggy bank gets a refund (a reward).
- If the idea is a dud, the fee is gone.
- If the piggy bank runs low, the AI is forced to become super careful and stop testing weak ideas. This ensures that, on average, only 5% of the final "winners" are actually fakes.

2. The First Wall: The "Haskell Monadic" Accountant

The first layer of the fortress is a strict accountant built into the AI's brain (using a programming language called Haskell).

The Metaphor: Imagine a video game where you cannot press the "Jump" button unless you have exactly the right amount of energy, and the game physically prevents you from pressing it if you don't.
How it works: The AI cannot even think about testing a hypothesis unless it first updates the piggy bank account. The code is written so that it is impossible to skip this step. If the AI tries to cheat the math, the system simply refuses to run the test. It's like a vending machine that won't give you a soda unless you insert the exact coins; you can't just grab the soda.

3. The Second Wall: The "OS-Level Sandbox"

The second layer protects against a different trick: the AI accidentally (or maliciously) peeking at the answer key while studying.

The Metaphor: Imagine a student taking a final exam. The "bad" way is to let the student study with the textbook open, then take the test. The "good" way is to lock the textbook in a safe in another room.
How it works: The AI writes code to test its ideas. The paper's system creates a digital "airlock" room (an OS-level sandbox) where the AI works.
- Inside this room, the "Answer Key" (the validation data) does not exist. It has been physically removed from the computer's view.
- Even if the AI tries to write code that says, "Go read the file at /secret/answers.txt," the door to that file is locked and the file isn't there. The AI literally cannot see the data it needs to cheat.
- The paper tested this with a "villain" AI that was explicitly told to cheat. The villain tried to read the file, but the airlock held firm. The leak rate was zero.

4. The "Math Proof" (The Blueprints)

You might ask, "How do we know this math actually works?"

The authors didn't just guess; they used a mathematical proof assistant (Lean 4) to prove that the "piggy bank" logic is sound.
They also used a specialized tool (SPARK/Ada) to prove that the computer's math (which sometimes rounds numbers off) wouldn't accidentally break the piggy bank.
The Result: They created a chain of trust that goes from a pure math theorem all the way down to the actual computer code running on the chip. It's like having an architect, a builder, and a safety inspector all sign off on a bridge, proving it won't collapse even if the wind blows.

The Results: What Happened?

The team ran simulations and real-world tests:

Without the fortress: The AI claimed 41% of its findings were real, but they were mostly fake.
With the fortress: The AI kept the error rate near the target of 5% (actually hitting 1.1% in tests).
Real-world check: When the AI found a real improvement in a machine learning model, the system let it through. When it found a fake one, the system stopped it.
The "Cheater" Test: When they tried to trick the system into reading the secret data, the "airlock" worked perfectly. The AI couldn't cheat.

Summary

This paper builds a statistical seatbelt for AI scientists. It combines a strict accounting system (so the AI can't overspend on false claims) with a locked-down room (so the AI can't peek at the answers). It proves mathematically that this system works, ensuring that when an AI says "I discovered something new," we can actually believe it.

Technical Summary: Structural Enforcement of Statistical Rigor in AI-Driven Discovery

Problem Statement
AI-Scientist systems, which utilize Large Language Models (LLMs) to generate, implement, and evaluate scientific hypotheses, face two critical threats to statistical integrity:

Uncontrolled Multiple Testing: When automated systems test thousands of hypotheses, naive application of a fixed significance threshold (e.g., $\alpha = 0.05$ ) leads to a massive inflation of the False Discovery Rate (FDR). Without correction, the majority of reported "discoveries" may be spurious.
Data Leakage and Biased P-values: In hybrid architectures where LLMs generate Python code, there is a risk that the generated code inadvertently accesses validation data during the optimization phase. This produces artificially low p-values that render any subsequent statistical correction (like FDR control) meaningless, as the underlying p-values are biased.

Existing online FDR control methods (e.g., LORD++) provide mathematical guarantees, but these rely on assumptions (such as the independence of future p-values from past decisions and the validity of the p-values themselves) that standard software implementations can silently violate.

Methodology: A Defense-in-Depth Architecture
The authors propose a functional architecture that enforces statistical rigor through two complementary layers: a macro-level Haskell embedded Domain-Specific Language (eDSL) and a micro-level declarative scaffold backed by OS-level isolation.

Macro-Level: The Research Monad (Haskell)
- Function: This eDSL enforces the LORD++ protocol structurally. It treats the significance level as an "error budget" (alpha-wealth).
- Mechanism: Using a StateT transformer over an ExceptT monad, the system threads the statistical state (wealth, discovery times, timestamps) through every operation.
- Enforcement: It is structurally impossible to test a hypothesis without updating the state. The type system ensures that thresholds are calculated based only on past information (predictability), preventing the code from "peeking" at future p-values. If a protocol violation occurs (e.g., a non-sequential timestamp), the computation halts immediately.
Micro-Level: Declarative Scaffolding and OS-Level Sandboxing
- Function: This layer ensures that the p-values fed into the Research Monad are unbiased and that the LLM-generated code cannot access validation data.
- Mechanism: The Haskell orchestrator generates a rigid Python harness before the LLM writes any domain logic. This harness:
  - Enforces Data Separation: The LLM's optimize function receives only the exploration data split. The validation split is loaded exclusively by the harness during the validation phase.
  - Standardizes Tests: The harness executes a paired sign-flip permutation test on held-out per-example losses, ensuring the null p-values are super-uniform.
  - OS-Level Isolation: The optimization step runs inside an unprivileged Linux user+mount namespace. The data directory within this namespace is a view containing only the exploration split. Consequently, the validation file is physically absent from the filesystem visible to the optimization code, making data leakage impossible regardless of the code's file-path attempts.
Formal Verification Chain
To bridge the gap between mathematical theory and floating-point implementation, the authors constructed a verified chain:
- Lean 4 Formalization: The LORD++ theorem is formalized in Lean 4, deriving the error budget and proving marginal and full FDR control. This formalization identifies four necessary conditions for an implementation:
  - (H1) Uniform/Super-uniform p-values.
  - (H2) Predictability of thresholds.
  - (H3) Independence of future p-values from past decisions.
  - (H4) Non-negativity of the wealth process.
- SPARK/Ada Verification: The arithmetic condition (H4) is verified using SPARK/Ada. The wealth update is implemented using multiplication rather than subtraction to guarantee non-negativity under IEEE 754 rounding errors. The GNATprove toolchain verifies that the wealth invariant holds for all steps, discharging 30 verification conditions without sorry (unproven claims).

Key Results

Simulation Study: In a Monte Carlo simulation of 2,000 hypotheses, a naive approach resulted in an empirical FDR of 41% against a 5% target. The proposed monadic LORD++ architecture controlled the FDR at 1.1%, demonstrating effective correction.
Case Studies (Wine Dataset):
- When using a flawed cross-validated t-test (which inflates significance due to overlapping folds), the system reported three false discoveries.
- When using the structurally enforced permutation test on disjoint splits, the system correctly identified that none of the hypotheses were significant, avoiding false positives while maintaining the ability to detect real effects when data was sufficient (demonstrated on a larger make_moons benchmark).
Adversarial Evaluation: In a security test where an LLM was explicitly prompted to locate and read validation data, the OS-level sandbox achieved a 0.00% leak rate. The validation file was physically inaccessible to the optimization process, whereas a prompt-only baseline resulted in a 100% leak rate.

Significance and Claims
The paper claims to present the first verified chain from theorem to floating-point implementation for an online FDR procedure. Its significance lies in:

Structural Enforcement: Moving beyond "asking" the LLM to be careful to making statistical errors structurally impossible via type systems and OS isolation.
Formal Rigor: Providing a machine-checked proof (Lean 4) of the FDR control theorem and a verified arithmetic implementation (SPARK) that accounts for IEEE 754 rounding errors, ensuring the mathematical guarantees hold in practice.
Defense-in-Depth: Addressing both the statistical accounting (macro) and the data integrity (micro) required for autonomous scientific discovery.

The authors note that while the architecture prevents data leakage and FDR inflation, it does not guarantee the semantic correctness of the domain-specific logic generated by the LLM (e.g., if the LLM chooses a scientifically unsound optimization strategy). However, it significantly reduces the damage surface of such errors by ensuring that even flawed experiments do not produce spurious statistical claims.