Learn-to-Distance: Distance Learning for Detecting LLM-Generated Text

1. Problem Statement

The rapid advancement of Large Language Models (LLMs) like GPT, Claude, and Gemini has created a critical challenge: distinguishing between human-written text and AI-generated text. Existing detection methods face significant limitations:

• Zero-shot methods (e.g., DetectGPT) often fail when the LLM generates text under unseen prompts or distribution shifts.
• Machine Learning (ML)-based methods often rely on fixed, hand-crafted distance metrics (e.g., N-gram distance, Levenshtein distance) in rewrite-based approaches. These fixed metrics do not adapt to the specific geometric properties of different target LLMs or datasets, limiting generalization.
• Adversarial Robustness: Many detectors degrade significantly when text is paraphrased or subjected to decoherence attacks.

The paper addresses the need for a detection algorithm that is robust to unseen prompts, adaptable to various LLM architectures, and resistant to adversarial manipulation.

2. Methodology: Learn-to-Distance (L2D)

The authors propose L2D, a novel rewrite-based detection framework that moves beyond fixed distance metrics by adaptively learning the distance function between original and rewritten text.

A. Theoretical Foundation (Geometric Framework)

The paper establishes a geometric intuition for rewrite-based detection:

• Subspace Projection: Human text ($H$) and LLM-generated text ($M$) are modeled as subspaces. LLM text is viewed as a projection of human text onto a constrained subspace $M$.
• Reconstruction Error: When an LLM rewrites a text, the "reconstruction error" (distance between original and rewritten text) is theoretically smaller for LLM-generated text than for human text. This is because human text lies further from the LLM's generation subspace.
• Optimal Distance: The authors prove that the optimal distance function should assign near-zero distance to pairs of LLM-generated texts and maximal distance to pairs involving human text. Crucially, this optimal function depends on the specific target LLM, justifying the need for learning rather than using a fixed metric.

B. The L2D Algorithm

The core innovation is replacing fixed metrics with a learnable distance function $d_\phi$.

1. Data Collection: The system collects a human corpus ($D_h$) and an LLM-generated corpus ($D_m$) from the target model.
2. Rewriting: For each text $X$, the system generates $K$ reconstructions $\hat{X}_1, \dots, \hat{X}_K$ using a rewrite model.
3. Distance Function: The distance is parameterized by a language model $p_\phi$:
   $$d_\phi(X_1, X_2) = \left| \frac{\log p_\phi(X_1)}{\text{len}(X_1)} - \frac{\log p_\phi(X_2)}{\text{len}(X_2)} \right|$$
   This formulation measures the discrepancy in normalized log-probabilities.
4. Optimization: The parameters $\phi$ are fine-tuned (using LoRA) to maximize the gap between the average reconstruction errors of human texts and LLM texts:
   $$\max_\phi \left( \mathbb{E}_{X \sim D_h}[d_\phi(X, R(X))] - \mathbb{E}_{X \sim D_m}[d_\phi(X, R(X))] \right)$$
5. Inference: A new text is classified as LLM-generated if its average reconstruction error falls below a learned threshold.

3. Key Contributions

• Methodological: Introduction of an adaptive distance learning framework for rewrite-based detection, replacing static metrics with a learnable, model-specific function.
• Theoretical:
  • Formal proof (Proposition 1) that human text has a higher expected reconstruction error than LLM text under a geometric projection model.
  • Proof of robustness to unseen prompts (Proposition 2).
  • Characterization of the optimal distance function (Proposition 3), demonstrating why fixed distances are suboptimal.
• Empirical: Comprehensive evaluation across 24 datasets, 7 target LLMs (including GPT-4o, Claude-3.5, Gemini-2.5, Llama-3), and 3 types of unseen prompts (rewrite, expand, polish).

4. Experimental Results

The authors conducted over 100 experimental settings, comparing L2D against 12 state-of-the-art baselines (including DetectGPT, Fast-DetectGPT, ImBD, and RAIDAR).

• Performance: L2D achieved the highest Area Under the Curve (AUC) in the vast majority of scenarios.
  • It outperformed the strongest baseline (ImBD) with relative improvements ranging from 54.3% to 75.4% across different target LLMs.
  • On average, L2D showed a 41.5% to 75.4% relative improvement over baselines.
• Generalization: L2D maintained high performance even when training and testing data involved different models (e.g., training on GPT-4, testing on GPT-3.5) or different domains.
• Ablation Study: Comparing L2D (learned distance) against a variant using a fixed pre-trained model (FD) showed that learning the distance function provided a 96% relative improvement, confirming the necessity of adaptation.
• Adversarial Robustness: Under paraphrasing and decoherence attacks, L2D remained robust, whereas other top-performing methods (like RAIDAR and ImBD) saw significant drops in AUC.
• Efficiency: The runtime of L2D is comparable to existing rewrite-based methods (RAIDAR), with only a slight increase due to the fine-tuning step, making it practical for deployment.

5. Significance

This paper represents a significant shift in LLM detection strategies. By moving from static, hand-crafted metrics to adaptive, learned distance functions, the authors address the fundamental issue of distribution shift caused by different LLMs and prompts.

• Theoretical Insight: It provides a rigorous geometric explanation for why rewrite-based methods work and how to optimize them.
• Practical Impact: The method offers a robust solution for detecting AI content in real-world scenarios where the generation prompt and model are unknown or vary frequently.
• Open Source: The authors have released a Python implementation, facilitating reproducibility and further research in the field.

In conclusion, Learn-to-Distance (L2D) sets a new state-of-the-art for passive LLM detection, demonstrating that adaptively learning the geometry of text reconstruction errors is superior to relying on fixed statistical measures.