🤖 machine learning

Calibration, Not Compilation: Detecting and Repairing Misspecified Probabilistic Programs Written by Language Models

This paper argues that for probabilistic programs generated by language models, statistical correctness is defined by calibration rather than compilation, demonstrating that Bayesian workflow-based detection and repair significantly outperform traditional unit testing and self-review methods in identifying and fixing statistical misspecifications.

Original authors: Jian Xu, Delu Zeng, John Paisley, Qibin Zhao

Published 2026-07-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jian Xu, Delu Zeng, John Paisley, Qibin Zhao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Core Problem: "Running" Doesn't Mean "Right"

Imagine you ask a very smart robot to write a recipe for a cake. The robot gives you a list of ingredients and steps. You follow the instructions, the oven works, the cake comes out of the oven, and it looks like a cake.

In the world of computer code, this is called "compiling and running." If the code runs without crashing, traditional software testers say, "Great! The program works."

But in the world of probabilistic programs (code used for statistics and data science), this is dangerous. A program can run perfectly and produce a cake, but if the recipe called for "salt" instead of "sugar," the cake will taste terrible. The code didn't crash, but the result is statistically wrong.

The authors call these "code-invisible bugs." They are errors that a computer can't see just by looking at the code or running it. They only show up when you look at the data the program produces.

The Old Way vs. The New Way

The Old Way (The Unit Test):
Traditionally, to check if code is good, we use "unit tests." These are like checking if the cake has the right shape and weight.

Does the program run? Yes.
Does it output numbers? Yes.
Result: "Pass!"

The paper shows that for statistical programs, this is useless. A program using the wrong math (like using a straight line to describe a curve) will still pass these tests. It's like saying a cake is perfect because it fits in the pan, even if it tastes like soap.

The New Way (The Calibration Oracle):
The authors propose a new verifier called the Calibration Oracle. Instead of just checking if the code runs, it checks if the story the code tells matches reality.

Think of it like a taste test or a weather forecast check:

Posterior Predictive Checks: The program predicts what the data should look like. The Oracle compares this prediction to the actual data. If the real data has huge spikes and the program predicts a flat line, the Oracle says, "You missed the mark."
Sampler Diagnostics: It checks if the program is struggling to find the answer (like a hiker getting lost in a foggy mountain). If the program is confused, it flags an error.
Held-out Density: It tests the program on data it hasn't seen before. If the program fails to predict new data accurately, it's misspecified.

The Experiment: Teaching the Robot to Fix Itself

The researchers tested this idea in three main ways:

1. Detection (Finding the Bug)
They created 200 fake scenarios where robots wrote statistical programs with hidden mistakes (like using the wrong type of math for the data).

The Result: The old "Unit Test" found 0% of the bugs. The new "Calibration Oracle" found 88% of them. It was like having a master chef who could taste the salt error, while the old method only checked the pan size.

2. Repair (Fixing the Bug)
They let Large Language Models (LLMs) try to fix their own broken programs. They gave the robots three types of feedback:

No Feedback: "Try again."
Unit Test Feedback: "Your code passed all tests. It's fine." (This actually made things worse because the robot thought it was already perfect and stopped trying to fix the hidden errors).
Calibration Feedback: "Your code runs, but your predictions don't match the data. The spread is too narrow."
The Result: The robots using Calibration Feedback fixed their mistakes much better. For some advanced models, the success rate jumped from 33% to 92%. The "Unit Test" feedback was harmful, acting like a false confidence booster that stopped the robot from fixing the real problem.

3. Real-World Test
They asked robots to write programs from scratch based on simple descriptions (no hints).

The Result: Even though 80-90% of the programs "ran," 15% to 47% were statistically wrong. The unit tests didn't catch a single one. The Calibration Oracle found the errors and helped the robots fix them, beating even other advanced AI reviewers.

Key Takeaways

Correctness is Calibration, Not Compilation: Just because a statistical program runs without crashing doesn't mean it's right. It's only right if its predictions are "calibrated" to the real world.
Tests Can Be Tricky: Telling a smart robot "all tests passed" can actually stop it from fixing deep, hidden errors. It creates a false sense of security.
The Sweet Spot: This new method works best for robots that are already quite smart but not yet perfect. It gives them the specific "taste test" feedback they need to improve.

In short: If you want a robot to write a statistical model, don't just ask it to "run the code." Ask it to "taste the cake" and make sure it matches the recipe. The paper proves that this "taste test" (Calibration) is the only way to catch and fix the invisible errors that standard code tests miss.

Technical Summary: Calibration, Not Compilation

Problem Statement

Large Language Models (LLMs) are increasingly used to draft probabilistic programs (e.g., in Stan, Pyro, NumPyro). While these programs often compile, run, and pass conventional unit tests, they can remain statistically misspecified. Common errors include using a Gaussian likelihood for heavy-tailed data, a Poisson distribution for over-dispersed counts, invalid prior supports, or pathological parameterizations (e.g., centered parameterization for hierarchical funnels).

The paper identifies a critical gap: standard software verification (compilation and unit tests) is blind to these statistical errors. A program that "runs and returns finite samples" may still produce a posterior that is poorly calibrated and fails to match held-out data. The authors term these code-invisible misspecifications. The central problem is that the standard "test oracle" (does the code run?) provides zero information about statistical correctness, potentially inducing false confidence that suppresses necessary repairs.

Methodology

The Calibration Oracle

The authors propose replacing the unit-test oracle with a Calibration Oracle derived from the Bayesian workflow. This oracle aggregates four families of diagnostics to verify if a fitted model is statistically correct:

Posterior Predictive Checks (PPC): Evaluates if the model can reproduce key data statistics (mean, SD, max, zero-counts) via tail probabilities.
Simulation-Based Calibration (SBC): Checks if the inference procedure is self-consistent with the model's prior by examining the rank distribution of prior draws against posterior draws.
Sampler Diagnostics: Monitors split- $\hat{R}$ , effective sample size (ESS), and divergent transitions to detect parameterizations with unreachable posterior geometry.
Held-out Predictive Density: Uses the log predictive density on held-out data as a strictly proper scoring rule to compare the model against a reference.

The oracle returns a verdict: the model is accepted only if it passes all thresholds (e.g., $\hat{R} < 1.05$ , low divergences, PPC $p$ -values not near 0 or 1, and high held-out density).

Experimental Setup

Benchmark: A dataset of 200 instances across 10 model families and 14 misspecification types. All instances compile and run.
Bug Taxonomy:
- Code-visible: Errors detectable by inspecting the source code under a neutral brief (e.g., hard-coded tiny noise).
- Code-invisible: Errors where the code is a faithful implementation of the brief but wrong for the specific data (e.g., wrong likelihood choice). This is the primary focus as test oracles fail here.
Repair Loop: An iterative process where an LLM drafts a program, the oracle evaluates it, and if uncalibrated, a structured diagnostic (e.g., "the spread of the data lies in the extreme tail") is fed back to the LLM for repair.
Models: Evaluated across 15 writers (Open source: DeepSeek, Qwen, Llama, Phi; API: GPT, Claude, Gemini).

Key Contributions

Detection Capability: The calibration oracle detects misspecification with an AUC of 0.97. On code-invisible bugs, it flags errors in 93% of cases at a 2% false-positive rate, whereas a unit-test oracle detects 0%.
Reference-Free Operation: The system does not require a hand-written correct program. A reference-free version (PPC + sampler + SBC) detects 62% of bugs. Adding a classical GLM baseline raises this to 68%, and an automated model search (LOO over a small library) reaches 78%. The 88% figure cited in the abstract assumes a known correct reference (an upper bound).
Repair Efficacy: Using calibration feedback significantly outperforms both "no feedback" and "unit-test feedback."
- Unit-test feedback is harmful: Telling an LLM that "all tests pass" suppresses repair, often resulting in lower fix rates than providing no feedback at all (e.g., Claude: 25% with test feedback vs. 75% with no feedback).
- Calibration feedback is effective: For strong-but-unsaturated models, calibration feedback dramatically improves fix rates (e.g., GPT-5.1: 33% $\to$ 92%; Claude: 75% $\to$ 100%).
Real-World Validation: On programs written from scratch for neutral briefs, 15–47% of runnable programs are statistically misspecified (unit tests catch none). Calibration-guided repair significantly outperforms baselines including LLM-as-judge review, Bayesian checklists, and data-summary self-debug.

Results

Detection: The oracle is nearly perfect for likelihood errors, dispersion issues, and zero-inflation. It struggles slightly with stochastic volatility and specific link errors that preserve data moments.
Localization: The oracle can localize the bug type with 75% top-2 accuracy, distinguishing between wrong likelihoods, bad parameterizations, and over-dispersion, though it sometimes confuses over-dispersion with prior-data conflict.
Non-Monotone Capability: The benefit of calibration feedback is non-monotone. It is negligible for models too weak to act on diagnostics and for models strong enough to repair blindly. It is most decisive for "strong-but-unsaturated" models.
Statistical Significance: Paired McNemar tests confirm that calibration feedback is significantly better than no feedback ( $p=4.4 \times 10^{-3}$ ) and unit-test feedback ( $p=6.5 \times 10^{-7}$ ). Unit-test feedback is significantly worse than no feedback ( $p=3.4 \times 10^{-3}$ ).

Significance and Claims

The paper argues that for probabilistic programs, correctness is calibration, not compilation.

Paradigm Shift: The right verifier for LLM-written probabilistic code is the Bayesian workflow itself, not a test suite.
Active Harm of Tests: Passing unit tests can actively mislead capable models into believing a misspecified model is correct, thereby suppressing the repair process.
Practical Impact: The calibration oracle provides a principled, reference-light foundation for trustworthy LLM-assisted probabilistic modeling. It bridges the gap between code generation and statistical validity, enabling capable LLMs to fix "code-invisible" errors that traditional software engineering tools cannot detect.

The authors note limitations: the oracle is not a universal certificate (it cannot detect causal structural errors if the predictive distribution is identical, and it struggles with high-dimensional SBC costs), but it is a robust verifier for the common statistical errors LLMs make.