$k$NNProxy: Efficient Training-Free Proxy Alignment for Black-Box Zero-Shot LLM-Generated Text Detection

1. Problem Statement

The proliferation of Large Language Models (LLMs) has created a critical need for detecting AI-generated text (LGT) to prevent misinformation and ensure digital trust.

• The Challenge: Existing detection methods fall into two categories:
  1. Learning-based: Require training on labeled data, which is costly and often fails to generalize to unseen models or domains (Out-of-Distribution/OOD).
  2. Zero-shot: Use a "proxy" LLM to simulate the unknown "source" LLM. These are training-free but rely heavily on the assumption that the proxy's output distribution closely matches the source's.
• The Gap: In real-world black-box scenarios, the source LLM is unknown. A single fixed proxy LLM (e.g., GPT-Neo) often fails to align with diverse proprietary source models (e.g., GPT-4, Claude, Gemini), leading to poor detection performance.
• Limitations of Current Solutions: Existing proxy alignment methods (e.g., DALD, Glimpse) attempt to bridge this gap but suffer from:
  • High Cost: Requiring supervised fine-tuning (DALD) or repeated, expensive API calls to retrieve top-$k$ probabilities (Glimpse).
  • Fragility: Susceptibility to silent API changes and lack of robustness under domain shifts (e.g., switching from academic to social media text).

2. Methodology

The authors propose kNNProxy, a training-free, query-efficient framework that repurposes the k-Nearest Neighbor Language Model (kNN-LM) mechanism as a domain adapter for a fixed proxy LLM.

A. Core Mechanism: kNNProxy

Instead of fine-tuning the proxy, kNNProxy augments it with a non-parametric datastore to correct token-level predictions.

1. Offline Datastore Construction:
   • A corpus reflecting the target source LLM is collected (via fixed-budget API queries or existing datasets).
   • A datastore $I = \{(h_n, y_n)\}$ is built, where $h_n$ is a context representation (encoded by the fixed proxy) and $y_n$ is the subsequent token.
   • This process is done once and requires no further training.
2. Inference & Alignment:
   • For a suspicious text, at each token position $i$, the system retrieves the $k$ nearest neighbors from the datastore based on the query context.
   • A retrieval-induced distribution ($\pi_{knn}$) is calculated based on the neighbors' next tokens, weighted by their distance.
   • Interpolation: The final aligned distribution $\hat{\pi}$ is a weighted blend of the fixed proxy's output ($\pi_{prx}$) and the retrieval output ($\pi_{knn}$):
     $$\hat{\pi}^{(i)} = \lambda_i \pi_{prx}^{(i)} + (1 - \lambda_i) \pi_{knn}^{(i)}$$
   • This aligned distribution is then fed into standard zero-shot detectors (e.g., Fast-DetectGPT, Binoculars).

B. Extension: Mixture of Proxies (MoP)

To address Out-of-Distribution (OOD) robustness (e.g., when the test text topic differs from the datastore topic):

• Architecture: Multiple domain-specific datastores are constructed (e.g., one for news, one for academic).
• Routing: A lightweight router (using sentence embeddings and kNN) classifies the input text's domain and routes it to the most compatible expert datastore.
• Benefit: Ensures domain-consistent retrieval, preventing the injection of misleading evidence from mismatched domains.

C. Theoretical Analysis & Adaptive Design

The authors derive a corpus-size-dependent approximation error bound for the retrieval-induced distribution. The error is decomposed into:

• Bias: Related to the distance between the query and neighbors (local smoothness).
• Variance: Related to the concentration of weights (effective neighborhood size).
• Adaptive Hyperparameters: Motivated by this bound, the method dynamically adjusts the neighbor size ($k$), temperature ($\tau$), and interpolation weight ($\lambda$) for each token.
  • Dense regions: Use larger $k$ and smoother weights to reduce variance.
  • Sparse regions: Use smaller $k$ to avoid bias from distant neighbors.
  • Reliability: If retrieval is unreliable (high error), the system leans more on the proxy ($\lambda \to 1$); if reliable, it leans on retrieval ($\lambda \to 0$).

D. Robust Aggregation

To handle outliers in token-level log-likelihoods, the method employs lower-tail clipping. Extreme low-likelihood tokens (often caused by poor proxy modeling) are clipped to a threshold $\gamma$ before averaging, preventing them from dominating the final detection score.

3. Key Contributions

1. Training-Free Alignment: Introduces kNNProxy, the first method to align a fixed proxy LLM with a source LLM using retrieval augmentation without fine-tuning or per-input API calls.
2. Domain Robustness (MoP): Proposes a Mixture of Proxies architecture that routes inputs to domain-specific experts, significantly improving performance on OOD text.
3. Theoretical Foundation: Derives an error bound for retrieval-based alignment, which directly motivates a novel token-wise adaptive hyperparameter strategy, eliminating the need for costly grid searches.
4. State-of-the-Art Performance: Demonstrates superior detection accuracy across diverse proprietary models and domains.

4. Experimental Results

The method was evaluated on Mix8 (8 proprietary LLMs, 4 domains) and DetectRL (including attack-augmented data).

• Detection Performance (AUROC):
  • kNNProxy + Binoculars achieved an average AUROC of 0.990 across 8 proprietary LLMs.
  • This represents a 6.45% improvement over the previous State-of-the-Art (SOTA) alignment method (Glimpse) and a 1.64% improvement over the unaligned Binoculars baseline.
  • It outperformed DALD (fine-tuning based) and Glimpse (API-based) significantly, especially in black-box settings where API access is restricted.
• Robustness:
  • Under OOD conditions (different topics or source models), kNNProxy maintained high F1 scores (~0.91), whereas baselines dropped significantly.
  • The MoP extension further improved OOD robustness, achieving an average F1 of 0.911 for the Binoculars+MoP variant.
• Source Attribution:
  • Beyond binary detection, kNNProxy was used for closed-set source attribution (identifying which LLM generated the text), achieving an average accuracy of 0.84 across 4 models.
• Ablation Studies:
  • Adaptive Hyperparameters: The token-wise adaptive strategy ($k, \tau, \lambda$) consistently outperformed fixed global settings.
  • Corpus Size: Performance scaled positively with datastore size, confirming the value of rich domain coverage.
  • Base Proxy: The method worked effectively with various base proxies (e.g., Falcon-7B-Instruct), showing it is not dependent on a specific underlying model.

5. Significance

• Practical Deployment: By eliminating the need for fine-tuning and expensive repeated API queries, kNNProxy offers a low-cost, scalable, and robust solution for real-world LGT detection.
• Resilience to API Changes: Unlike Glimpse, which breaks when APIs stop returning top-$k$ probabilities, kNNProxy relies only on text generation, making it future-proof against API policy changes.
• Theoretical Insight: The work bridges the gap between retrieval-augmented generation (kNN-LM) and forensic detection, providing a theoretical framework for understanding and optimizing retrieval-based alignment.
• Generalizability: The approach is not limited to detection; it extends to source attribution and could be adapted for other tasks requiring distribution alignment between a fixed model and a target distribution.

In summary, kNNProxy solves the critical "alignment gap" in black-box zero-shot detection by using a lightweight, retrieval-based adapter that is training-free, domain-adaptive, and theoretically grounded, setting a new benchmark for LLM-generated text detection.