Pre-Registering the Detectable Effect: A Paired-MDE Budget for 4-bit Quantization Benchmarks, with a Pilot Audit

Technical Summary: Pre-Registering the Detectable Effect in 4-bit Quantization Benchmarks

Problem Statement
Current practices in evaluating post-training quantization (specifically 4-bit NF4) for Large Language Models (LLMs) often rely on reporting single accuracy numbers per model-benchmark cell. This approach elides a fundamental statistical question: what is the smallest quantization effect that the evaluation protocol can reliably detect?

Standard benchmarks often use small subsamples (e.g., $n=100$ items) split into non-overlapping subsets. The observed variance across these splits is frequently interpreted as "benchmark unreliability" or a failure of the quantization method. However, the authors argue that much of this variance is simply binomial sampling noise inherent to small $n$, rather than a property of the model or the quantization scheme. Without an ex-ante guarantee on the Minimum Detectable Effect (MDE), claims that a quantization method "preserves" performance may be under-powered, conflating a small population effect with an inability to detect it.

Methodology
The paper proposes a framework to convert benchmark reliability into a pre-registerable budget line using a conservative paired-MDE bound.

1. Theoretical Derivation (Paired-MDE Bound):
   The authors adapt classical paired-binary sample-size calculations (Miettinen, 1968) to the context of FP16 vs. NF4 comparisons. They define:

   • $m$: The paired item count (either $n$ for a single split or $kn$ for an aggregate of $k$ splits).
   • $\rho_d$: The per-example disagreement rate ($Pr(D_i \neq 0)$), where $D_i$ is the difference in correctness between FP16 and NF4 on item $i$.
   • $\delta$: The population effect size ($|E[D_i]|$).

   They derive a conservative upper bound for the MDE ($\delta^*$) at significance level $\alpha$ and power $1-\beta$:
   $$\delta^*(m, \rho_d) \leq (z_{1-\alpha/2} + z_{1-\beta}) \sqrt{\frac{\rho_d}{m}}$$
   This bound assumes a worst-case disagreement rate $\rho_d$ (pre-registered or conservatively bounded) and treats the variance as $\rho_d/m$. It serves as a "budget" line: effects smaller than $\delta^*$ are not guaranteed to be detectable by the design, regardless of the specific model.

2. Empirical Pilot Audit:
   The authors conduct a pilot audit on a $4 \times 4$ grid:

   • Models: OPT-2.7B, Pythia-2.8B, Llama-2-7B, Mistral-7B.
   • Benchmarks: MMLU, ARC-Easy, WinoGrande, HellaSwag.
   • Design: $k=5$ non-overlapping splits of $n=100$ items each ($m=500$ aggregate).
   • Prompt Sensitivity: A parallel study on MMLU using three distinct prompt templates ($n=50$ each) to measure prompt-induced variance.

3. Diagnostic Metrics:

   • Binomial Reference: Comparing observed cross-split standard deviations ($\hat{\sigma}_{split}$) against the theoretical binomial reference $\sqrt{\hat{p}(1-\hat{p})/n}$.
   • Quantization Reliability Index (QRI): A signal-to-noise heuristic defined as $|\hat{\Delta}| / \hat{\sigma}_{split}$ (and a combined variant including prompt variance) to flag cells where noise dominates the signal.

Key Results

• Dominance of Sampling Noise: On the $n=100$ subsamples, 25 of 32 observed cross-split standard deviations fell within $\pm 1.5$ percentage points (pp) of the binomial reference. This suggests that much of the reported "unreliability" is simply small-sample binomial noise, not quantization-specific instability.
• Detectability Limits:
  • Assuming a planning disagreement rate of $\rho_d = 0.10$, the aggregate MDE ($m=500$) is $\approx 4.0$ pp.
  • Assuming $\rho_d = 0.05$, the MDE tightens to $\approx 2.8$ pp.
  • Observation: All observed NF4–FP16 accuracy deltas for the 7B models (ranging from -0.4 to -0.8 pp) fell below both MDE thresholds.
  • Borderline Case: The only cell approaching the bound was OPT-2.7B on WinoGrande ($|\Delta| = 3.2$ pp). This effect is below the MDE at $\rho_d=0.10$ but above it at $\rho_d=0.05$, illustrating that the detectability of this specific result depends entirely on the unmeasured planning value of $\rho_d$.
• Prompt Variance Confound: On MMLU, prompt-template variations caused accuracy swings of 2–10 pp. This range meets or exceeds the largest observed quantization delta (3.2 pp). Consequently, a quantization audit that does not fix the prompt template absorbs template variance into its noise floor, potentially masking or fabricating effects.

Contributions

1. Conservative Paired MDE Bound: A closed-form inequality (Eq. 2) that allows benchmark designers to pre-register a detectability budget based on sample size and a prior disagreement rate.
2. Empirical Audit & Binomial Reference: A pilot study demonstrating that on standard subsample sizes ($n=100$), cross-split variance is largely explained by binomial sampling, not model-specific noise.
3. Quantization Reliability Index (QRI): A descriptive signal-to-noise metric to identify cells where evaluation noise dominates the quantization signal.
4. Pre-Registration Template: A five-line template for researchers to commit to MDE targets, disagreement priors, and paired statistics before running benchmarks.

Significance and Claims
The paper does not claim to prove that 4-bit quantization is universally perfect or that specific models fail. Instead, it argues that benchmark comparisons between quantized and full-precision models are routinely under-powered.

The authors assert that by making the MDE explicit, the community can reframe "small-effect" narratives. If an observed delta is below the pre-registered MDE, the correct conclusion is not that the effect is "small" or "negligible," but that it is "not power-distinguishable at this sample size."

The study highlights two critical prerequisites for meaningful precision comparisons:

1. Sample Size: Current subsample sizes ($n=100$) are insufficient to resolve sub-percentage-point effects without assuming extremely low disagreement rates.
2. Prompt Control: Prompt template variance must be fixed or swept across, as it can exceed the magnitude of the quantization effect itself.

The work serves as a methodological intervention, urging the field to adopt ex-ante guarantees (similar to Clinical Trial Budgets) to distinguish between a true lack of quantization impact and a failure of the evaluation protocol to detect it.