Technical Summary: WAVEDETECT

Problem Statement

The rapid advancement of Large Language Models (LLMs) has led to machine-generated text (MGT) that is increasingly indistinguishable from human-written text (HWT) in terms of fluency and semantic coherence. This capability poses significant risks to information integrity, including academic dishonesty and the proliferation of disinformation. Existing detection methods face three critical limitations:

1. Adversarial Vulnerability: Detectors relying on surface-level semantic artifacts (e.g., perplexity, entropy) are brittle and easily evaded by adversarial perturbations such as paraphrasing, synonym substitution, or character-level obfuscation.
2. Temporal Instability: The rapid iteration of LLM architectures renders detectors obsolete quickly, as models trained on legacy architectures often fail to generalize to unseen, next-generation models.
3. Domain Generalization: Detectors trained on general corpora frequently fail when applied to specialized domains (e.g., law, medicine) due to overfitting on domain-specific vocabulary rather than fundamental generation signatures.

Current approaches, whether zero-shot (relying on probability curvature) or supervised (training classifiers on token sequences), struggle to capture the intrinsic, invariant differences between human and machine generation, often focusing on the time-domain representation of text where the boundary between human and machine semantics is blurred.

Methodology

The authors propose WAVEDETECT, a framework that reformulates MGT detection as a signal processing task within the time-frequency domain. Instead of analyzing static token probabilities or raw sequences, WAVEDETECT models the generated output as a probability signal and applies a Continuous Wavelet Transform (CWT) to extract "spectral fingerprints."

Core Workflow

1. Signal Extraction: A surrogate language model ($M_\theta$, specifically Qwen2.5-0.5B-Base) maps a discrete sequence of tokens $X = \{x_1, \dots, x_N\}$ into a continuous probability signal $p_t = P_{M_\theta}(x_t | x_{<t})$.
2. Spectral Hypothesis: The framework hypothesizes that human writing, characterized by high-entropy and stochastic selection, produces probability signals with irregular "spikes" and chaotic transients. In contrast, machine generation, constrained by decoding strategies (e.g., top-k, top-p, repetition penalties), produces artificially smoothed signals with structured, periodic rhythmic patterns that suppress high-frequency transients.
3. Continuous Wavelet Transform (CWT): To capture both global spectral features and local transient features, the probability signal $p(t)$ is decomposed using the Morlet wavelet. This transforms the signal into a time-frequency representation (spectrogram) $W(s, \tau)$, defined as the convolution of the signal with scaled and translated wavelets. This step preserves temporal localization information that global Fourier methods lose.
4. Feature Extraction & Classification: The resulting magnitude spectrum $|W|$ is treated as a single-channel image and fed into a lightweight Convolutional Neural Network (CNN), specifically a ResNet-18 backbone, to classify the source as human or machine.
5. Optimization Strategy: A two-stage training protocol is employed:
   • Warm-up: The surrogate model is frozen while the CNN learns to extract robust features from the pre-trained surrogate's output.
   • Joint Training: The surrogate model is unfrozen to refine the probability distribution, allowing the system to jointly optimize the signal generation and spectral feature extraction.
   • Loss Function: A Weighted Cross-Entropy (WCE) loss is used to address class imbalance and maximize the decision margin.

Key Contributions

1. Novel Framework: Introduction of WAVEDETECT, which leverages CWT to extract robust time-frequency features, characterizing the temporal dynamics of token probabilities beyond surface text features.
2. State-of-the-Art Performance: The method achieves superior accuracy and robustness across three rigorous evaluation dimensions: adversarial attacks, model evolution, and domain shifts.
3. Spectral Disentanglement: Visualizations (Class Activation Maps and t-SNE) demonstrate that human and machine text distributions can be effectively disentangled in spectral space, revealing that the CNN learns scale-dependent structures (high-frequency transients for humans vs. low-frequency regularities for machines) rather than absolute spectral energy.

Experimental Results

The framework was evaluated on three datasets: RAID (adversarial robustness), EvoBench (temporal stability), and Domain-Shift (out-of-distribution generalization).

• Adversarial Robustness (RAID-test): WAVEDETECT achieved an average AUROC of 0.9785, significantly outperforming the best zero-shot baseline (Binoculars, 0.8414) and supervised baseline (RADAR, 0.8277). Crucially, it maintained high performance (>90% accuracy) against sophisticated attacks like zero-width space injection and homoglyph substitution, where other methods suffered significant degradation. At a strict 0.1% False Positive Rate, WAVEDETECT achieved a True Positive Rate of 54.70%, nearly 19 percentage points higher than the next best method.
• Temporal Stability (EvoBench): When tested on evolving versions of closed-source models (e.g., GPT-4o variants, Claude-3.5), WAVEDETECT demonstrated superior stability with the lowest standard deviation and maximum performance fluctuation ($\Delta$) compared to baselines, proving its ability to generalize to unseen model architectures.
• Domain Generalization: On specialized domains (Medical and Legal) unseen during training, WAVEDETECT achieved the highest average AUROC (0.9560). Unlike traditional methods that dropped to near-random performance in these domains, WAVEDETECT maintained consistent performance, confirming that its spectral features are "topic-agnostic."
• Efficiency: The method is computationally efficient, requiring only a single forward pass of a 0.5B surrogate model and a lightweight CNN (11.7M parameters), making it suitable for large-scale deployment compared to heavier baselines.

Significance and Claims

The paper claims that WAVEDETECT validates spectral analysis as a promising paradigm for MGT detection. By shifting from semantic analysis to spectral analysis, the framework captures intrinsic "spectral fingerprints" that remain invariant to semantic paraphrasing, domain shifts, and model evolution. The authors assert that these rhythmic probability fluctuations are fundamental to the generative mechanisms of LLMs, offering a more robust and generalizable solution than methods relying on surface-level artifacts. The work suggests that future detectors should focus on the time-frequency dynamics of probability signals rather than static token sequences.

Limitations

The authors acknowledge that the current evaluation relies on the RAID dataset, which contains relatively older or weaker LLMs compared to the latest generation. They note that detectors trained on modern LLMs might perform even better, but conversely, adversarial attacks on modern MGTs may present new challenges. Furthermore, the specific mechanisms of the wavelet transform (e.g., the optimal choice of mother wavelet and scale ranges) require further exploration.