DetectZoo: A Unified Toolkit for AI-Generated Content Detection Across Text, Audio, and Image Modalities

Technical Summary: DetectZoo

Problem Statement
The rapid advancement of generative AI has blurred the distinction between human- and machine-generated content, creating a critical need for robust detection methods across text, image, and audio modalities. However, the current research landscape is fragmented. Existing detectors are typically released as standalone codebases with incompatible preprocessing pipelines, evaluation protocols, and metrics. This fragmentation creates three primary obstacles: (1) a lack of fair comparison due to differences in tokenization, preprocessing, and threshold selection; (2) high barriers to reproducibility due to missing dependencies and undocumented hyperparameters; and (3) an absence of a cross-modal perspective, as detectors for different modalities exist in isolated ecosystems.

Methodology and Toolkit Architecture
To address these challenges, the authors introduce DetectZoo, a unified, open-source, and extensible toolkit designed to provide a standardized interface for AI-generated content detection. The framework is built on three core principles: reproducibility, accessibility, and extensibility.

• Unified API: DetectZoo provides a single factory function (load_detector) and a standardized predict() method that accepts diverse inputs (strings, image paths, audio files) and returns a consistent DetectionResult object containing binary labels, anomaly scores, and confidence metrics. This abstraction eliminates the need for users to manage modality-specific differences.
• Standardized Evaluation Pipeline: The BenchmarkEvaluator module systematically iterates through datasets, aggregating scores and ground-truth labels to compute both threshold-independent metrics (AUROC, Precision-Recall AUC, Average Precision, Equal Error Rate) and threshold-dependent metrics (Accuracy, F1, Precision, Recall).
• Modular Design: The toolkit uses a registration-based pattern where new detectors and datasets are introduced by subclassing core abstractions (e.g., BaseDetector) and annotating them with decorators. This allows for seamless integration of new components while maintaining a uniform interface.
• Automated Data Management: DetectZoo includes native loaders for 22 benchmark datasets across text, image, and audio. These loaders automatically download, cache, and format data, ensuring consistent experimental conditions.

Key Contributions

1. Unified Multi-Modal API: A standardized architecture spanning text, image, and audio detection methods, significantly reducing the engineering overhead for benchmarking novel algorithms against established baselines.
2. Standardized Benchmark Datasets: Integration of 22 diverse public datasets, including HC3, RAID, ForenSynths, and ASVspoof 2019, ensuring robust and reproducible evaluation environments.
3. Comprehensive Baseline Implementations: Reference implementations of 61 state-of-the-art detectors, covering paradigms from zero-shot statistical anomaly detection to supervised deep representation learning. These include:
   • Text (36 detectors): Statistical methods (e.g., log-likelihood, entropy), perturbation-based methods (e.g., DetectGPT, Fast-DetectGPT), multi-model techniques (e.g., Binoculars), representation-based approaches (e.g., PHD), and supervised models (e.g., RoBERTa-based classifiers).
   • Image (15 detectors): Artifact-based methods (e.g., CNNSpot), frequency/structure-aware methods (e.g., FreqNet, SAFE), reconstruction-based methods (e.g., AEROBLADE), and CLIP-based methods (e.g., UnivFD, FatFormer).
   • Audio (10 detectors): End-to-end waveform methods (e.g., RawNet2, AASIST) and pretrained feature methods leveraging foundation models (e.g., Whisper, Wav2Vec2, HuBERT).
4. Open-Source Extensibility: The complete toolkit is released as a PyPI package (detectzoo) with modular optional dependencies, fostering community-driven advancements.

Results and Empirical Findings
The paper validates the toolkit’s reproducibility by reproducing results from original authors for selected detectors in each modality. Key empirical findings from the benchmarking include:

• Text Modality: Detection difficulty is primarily driven by task semantics. Tasks that preserve the surface form of original text (e.g., rewriting, polishing) cause statistical detectors to perform near chance levels, whereas direct generation is easier to detect. The source Large Language Model (LLM) is a systematic confound; GPT-4o and Qwen2-7B texts are consistently the hardest to detect, while Llama-3 texts are the easiest.
• Image Modality: CLIP-based and hybrid methods (e.g., FatFormer, AIDE, SAFE) demonstrate the strongest generalization across intra- and cross-architecture settings. Unlike earlier artifact-based detectors, their performance degrades less when transferred to unseen generators. Training-free reconstruction methods (e.g., AEROBLADE) show inconsistent behavior, performing poorly on GAN benchmarks but strongly on diffusion-heavy datasets.
• Audio Modality: Large-scale multilingual pretraining yields robust Out-of-Distribution (OOD) detection. Foundation models such as AntiDeepfake Wav2Vec2-Large and HuBERT-XLarge achieve superior Equal Error Rates (EER) on ASVspoof 2019 despite not being trained on the dataset, demonstrating the transferability of speech representations for zero-day spoofing detection.

Significance and Limitations
DetectZoo is positioned as a research infrastructure contribution rather than a new detection algorithm. Its significance lies in making detection methods accessible, comparable, and reproducible, similar to existing toolkits like CLIMB or RobustBench. By consolidating fragmented implementations, it enables systematic analysis and accelerates the development of robust, generalizable detection techniques.

The authors acknowledge several limitations:

• The toolkit currently focuses on evaluation and does not provide unified training pipelines for supervised detectors.
• Coverage is uneven, with stronger representation in text than in audio, and no support for video or other emerging domains.
• Some detectors depend on external reference models that cannot be redistributed, potentially limiting strict reproducibility.
• Benchmark datasets represent static snapshots of generation behavior and may not fully capture the characteristics of newer models.
• The authors recognize the dual-use nature of the toolkit, noting that malicious actors could theoretically use it to optimize evasion tactics, but argue that transparent, reproducible benchmarking is essential for advancing robust security solutions.