A Unified Detection Framework for AI-Related Content and Artifacts

Technical Summary: A Unified Detection Framework for AI-Related Content and Artifacts

Problem Statement
The rapid deployment of Artificial Intelligence (AI), particularly Large Language Models (LLMs) and generative models, has introduced significant oversight challenges. These include the proliferation of AI-generated text, hallucinated or factually inconsistent outputs, watermarked content, and adversarial perturbations. Existing detection methods are often task-specific, relying on tailored features, supervised training data, or access to internal model parameters (white-box). Furthermore, current approaches struggle to provide a unified statistical treatment for heterogeneous reference samples that may arise from multiple classes, sources, or domains. There is a need for a robust, black-box detection framework that can characterize "positive" (normal) samples effectively, even when those samples are contaminated or belong to multiple distinct classes.

Methodology
The authors propose a unified black-box detection framework based on Mahalanobis Distance Scores (MDS). The core philosophy is to characterize the distribution of positive samples (e.g., human text, factual statements, clean images) in a learned representation space and identify negative samples (e.g., AI-generated text, hallucinations, adversarial examples) as those deviating significantly from this characterization.

The methodology proceeds in two main stages:

1. Feature Extraction: A pretrained model is used to extract deep latent representations from the input data.
2. Robust Covariance Estimation: Since positive samples may contain contamination (outliers) or consist of multiple classes with shared and specific structures, the authors develop robust estimators for the mean vector and covariance matrix.
   • Single-Class Setting: They employ Casewise Minimum Covariance Determinant (MCD) to handle entire outlier observations and Cellwise MCD to handle contamination at the feature-entry level (where only specific dimensions of a sample are corrupted).
   • Multi-Class Setting: To handle positive samples from multiple classes (e.g., different image categories or text domains), the authors propose Joint Estimators. They introduce a reparameterization technique where the covariance matrix of each class $k$ is decomposed as $\Sigma^{(k)} = \Theta \odot \Gamma^{(k)}$. Here, $\Theta$ represents a shared covariance pattern across all classes, while $\Gamma^{(k)}$ captures class-specific variations. This allows the model to borrow strength across classes while preserving heterogeneity.
   • Optimization: The authors develop efficient optimization algorithms for these joint estimators. For the joint casewise MCD, they use an alternating optimization approach similar to the Fast-MCD algorithm. For the joint cellwise MCD, they utilize an Expectation-Maximization (EM) procedure to handle the binary cellwise weights and continuous parameters.

Key Contributions

1. Unified Framework: The paper proposes a general black-box detection framework applicable to diverse AI-related tasks, including LLM-generated text detection, hallucination detection, watermark detection, and adversarial example detection.
2. Joint Estimation Methods: The authors develop joint estimation methods for multiple covariance matrices in low-dimensional settings (after dimensionality reduction). These methods cover both casewise and cellwise contamination mechanisms and allow positive samples to contain multiple classes with both shared and class-specific covariance structures.
3. Theoretical Guarantees:
   • Convergence: The paper provides efficient optimization algorithms and proves their convergence, showing that the objective values decrease monotonically and converge to finite limits.
   • Robustness: The authors define a suitable breakdown point for joint estimators and prove that the proposed methods possess high breakdown point properties, meaning they remain stable even when a significant fraction of the data is contaminated.

Experimental Results
The framework was evaluated on four real-world tasks:

• LLM-Generated Text Detection: Using the HC3 dataset and various LLMs (GPT-5.4, Gemini), the proposed methods (JCASEMCD and JCELLMCD) achieved competitive ROC AUC scores (e.g., 0.969 on GPT-5.4), often outperforming or matching state-of-the-art baselines like DetectGPT, particularly in specificity and precision.
• Watermark Detection: The framework was tested on text generated with Gumbel-Max and Inverse-Transform watermarks. While specialized hypothesis-testing methods performed best when the watermark rule was known, the proposed framework offered a more balanced precision-recall trade-off and demonstrated superior generalization when the watermarking mechanism was unknown or absent.
• Hallucination Detection: Evaluated on reference-free and reference-based benchmarks (HaluEval, wiki_bio), the cellwise variant (JCELLMCD) achieved the best overall performance among the proposed methods, outperforming DeBERTa-based baselines by alleviating low-specificity issues.
• Adversarial Example Detection: Tested on CIFAR-10, CIFAR-100, and ImageNet with various attacks (FGSM, BIM, CWL2, DeepFool). The framework matched or exceeded baselines in simpler settings but showed limitations in complex, high-dimensional multi-class scenarios (e.g., CIFAR-100 under CWL2), likely due to the instability of joint covariance estimation in those specific conditions.

Significance and Claims
The paper claims that combining deep representations with robust joint covariance estimation and Mahalanobis distance scores provides a flexible and statistically principled approach for detecting diverse AI-related anomalous samples. The authors emphasize that their approach is particularly valuable for black-box scenarios where internal model information is inaccessible. By addressing the challenge of heterogeneous positive samples through joint estimation, the framework offers a unified solution that reduces the need for task-specific retraining or data curation. The theoretical establishment of high breakdown points ensures that the detection mechanism remains reliable even when the training data for the "positive" class is not perfectly clean. The authors conclude that while the framework is effective, future work is needed to relax normality assumptions and improve stability in high-dimensional settings.