Technical Summary: AdaDetectGPT

Problem Statement

The paper addresses the critical challenge of distinguishing between text authored by humans and text generated by Large Language Models (LLMs). While existing state-of-the-art detectors rely on statistics derived from the log-probabilities (logits) of observed text evaluated against a source LLM's distribution, the authors argue that relying solely on raw log-probabilities is sub-optimal. Current methods often fail to fully exploit the statistical differences between human and machine-generated distributions, particularly in complex scenarios involving different datasets and model architectures.

Methodology: AdaDetectGPT

The proposed method, AdaDetectGPT, is an adaptive classifier designed to enhance existing logits-based detectors (specifically Fast-DetectGPT) by learning a "witness function" from training data.

1. Statistical Framework

The method operates under two settings:

• White-box: The source LLM used to compute logits is identical to the target LLM generating the text.
• Black-box: The source LLM is an open-source approximation of the target closed-source model.

The core statistic $T_w(X)$ is constructed as a normalized sum of transformed log-probabilities:
$$T_w(X) := \frac{\sum_{t}[w(\log q_t(X_t|X_{<t})) - \mathbb{E}_{\tilde{X}_t \sim q_t} w(\log q_t(\tilde{X}_t|X_{<t}))]}{\sqrt{\sum_{t} \text{Var}_{\tilde{X}_t \sim q_t}(w(\log q_t(\tilde{X}_t|X_{<t})))}}$$
Here, $w: \mathbb{R} \to \mathbb{R}$ is a one-dimensional witness function applied to the log-probabilities. Unlike Fast-DetectGPT, which uses the identity function (raw logits), AdaDetectGPT learns $w$ to maximize detection power.

2. Threshold Selection via Martingale Theory

A key theoretical contribution is the derivation of the classification threshold. By modeling the token generation process as a time series and applying the Martingale Central Limit Theorem (MCLT), the authors establish that under the null hypothesis (text generated by the LLM), the statistic $T_w(X)$ converges to a standard normal distribution as the sequence length $L \to \infty$.

• This allows for the selection of a threshold $c = z_\alpha$ (the $\alpha$-quantile of the standard normal distribution) to strictly control the False Negative Rate (FNR) at a desired level $\alpha$.

3. Learning the Witness Function

The primary challenge is that maximizing the True Negative Rate (TNR) for a fixed FNR typically yields a witness function dependent on the specific FNR level $\alpha$. To overcome this, the authors:

• Derive a lower bound on the TNR that separates the effects of $\alpha$ and the witness function $w$.
• Show that maximizing this lower bound is equivalent to maximizing a population-level quantity $T^{(2)*}_w$, which is independent of $\alpha$.
• Implement this optimization using a linear function class over B-spline basis functions. The optimization reduces to solving a system of linear equations ($\Sigma \beta = \psi$), making the training process computationally efficient.

Key Contributions

1. Adaptive Detection: The introduction of a learnable witness function that transforms raw logits, empirically shown to better distinguish human and machine text than raw logits alone.
2. Statistical Guarantees: The paper provides finite-sample error bounds for the True Positive Rate (TPR), False Positive Rate (FPR), True Negative Rate (TNR), and FNR. Specifically, it proves that as the training sample size $n$ and sequence length $L$ increase, the classifier's performance converges to that of an oracle classifier with access to the optimal population witness function.
3. Theoretical Foundation for Thresholding: The application of MCLT to justify the use of normal approximation for FNR control, a feature often lacking in previous statistical detectors.
4. Efficient Optimization: The reduction of the witness function learning problem to a simple linear system, avoiding complex non-convex optimization.

Experimental Results

The authors conducted extensive numerical studies across five datasets (SQuAD, WritingPrompts, XSum, Yelp, Essay) and various LLMs (GPT-2, OPT, GPT-Neo, GPT-J, GPT-NeoX, Qwen2.5, Mistral, LLaMA3).

• White-box Performance: AdaDetectGPT consistently outperformed eight state-of-the-art baselines (including DetectGPT, Fast-DetectGPT, and DNAGPT). It achieved Area Under the Curve (AUC) improvements ranging from 12.5% to 37% over the best baseline (Fast-DetectGPT).
• Black-box Performance: When detecting text from advanced closed-source models (GPT-4o, Claude-3.5, Gemini-2.5-Flash) using an open-source proxy, AdaDetectGPT maintained superior performance, with improvements up to 20% over Fast-DetectGPT.
• Robustness: The method demonstrated resilience against adversarial attacks, specifically paraphrasing and decoherence, outperforming baselines by up to 10% and 85% respectively in specific black-box scenarios.
• Efficiency: Training the witness function required less than one minute and under 0.5 GB of memory.

Significance and Claims

The paper claims to fill a gap in the literature regarding the systematic statistical analysis of logits-based detectors. While previous works focused on empirical performance, this work provides rigorous statistical guarantees on error rates.

The authors position AdaDetectGPT at the intersection of statistics-based and machine learning-based methods. It retains the interpretability and data-efficiency of statistical methods (relying on log-probabilities) while leveraging the adaptability of machine learning (learning a witness function) to achieve superior detection power. The method is presented as a robust, theoretically grounded solution for the growing need to detect LLM-generated content without relying on model-specific watermarks or black-box training data.