Detecting Diffusion-Generated Time Series Under Generator Shift

Technical Summary: Detecting Diffusion-Generated Time Series Under Generator Shift

Problem Statement
The boundary between real and diffusion-generated time series is increasingly difficult to distinguish, posing risks for data integrity in critical domains like healthcare and finance. While synthetic data generation via diffusion models (e.g., SSSD, TSDiff) has advanced, the detection of such synthetic data remains underexplored, particularly when the generator is unknown. Existing detection methods often rely on assumptions valid in the image domain—specifically, the availability of a "universal" reconstruction prior or direct access to the generator's internal distribution. However, these assumptions do not hold for time series, where models are typically trained on narrow distributions with unique reverse dynamics. The core challenge addressed is detecting diffusion-generated time series under generator shift (Out-of-Distribution or OOD scenarios), where the detector must identify samples from generators unseen during training.

Methodology
The paper evaluates two distinct detection strategies:

1. White-Box Reconstruction-Based Detection:
   Adapted from image domain methods (specifically DIRE), this approach requires access to a reference diffusion model ($G^*$) trained on the same distribution as the target generator. The detector uses $G^*$ to perform DDIM inversion and reconstruction on an input time series $x_0$. The reconstruction error $d_{G^*}(x_0) = (x_0 - R_{G^*}(x_0))^2$ serves as the detection cue. The hypothesis is that synthetic samples from $G^*$ will have low reconstruction errors, while real samples will have high errors.

   • Limitation: In the time series domain, no generic generator exists. When applied to samples from an unseen generator ($G_i$), the reconstruction model $G^*$ attempts to map the input toward its own learned distribution ($p_{G^*}$) rather than preserving the input structure. Consequently, the reconstruction errors for real samples and samples from unseen generators collapse into indistinguishable distributions, causing the decision boundary to vanish.

2. Black-Box Classifier-Based Detection:
   This approach bypasses reconstruction entirely. It treats detection as a binary classification problem where a standard time series classifier is trained directly on raw signals labeled as "real" or "synthetic" (generated by a single known generator, $G^*$). The classifier learns to identify synthesis artifacts directly from the raw data without requiring access to the generator's internal mechanics or a reconstruction prior. The study utilizes off-the-shelf architectures, primarily Disjoint-CNN, alongside InceptionTime and LITE, trained on real data paired with samples from SSSD.

Key Contributions

1. First Systematic Exploration: This work provides the first comprehensive comparison of white-box and black-box detection strategies for diffusion-generated time series.
2. Identification of Domain-Specific Failure: The authors demonstrate that reconstruction-based detection, successful in images due to universal priors (e.g., Stable Diffusion), fails in time series under generator shift. They show that time series diffusion models lack a universal reconstruction prior; instead, they learn model-specific reverse dynamics that cause reconstruction errors to overlap for real and unseen synthetic data.
3. Validation of Black-Box Viability: The study establishes that a simple off-the-shelf classifier trained on a single generator can effectively generalize to unseen generators, outperforming the white-box approach significantly in OOD settings.

Experimental Results
The evaluation was conducted across ten benchmark datasets, three sequence lengths (32, 64, 128), and four diffusion-based generators (SSSD, TSDiff, Diffusion-TS, WaveStitch).

• In-Distribution (ID): The white-box approach (DIRE) performs well (Average F1: 91.9), though slightly below the black-box classifier (Disjoint-CNN), which achieves near-perfect performance (F1 > 94).
• Out-of-Distribution (OOD): The performance gap widens drastically.
  • White-Box (DIRE): Fails under generator shift, achieving an average OOD F1 of 64.9 and a True Positive Rate at 1% False Positive Rate (TPR@1%FPR) near 0.00 for most unseen generators. The reconstruction errors for real and synthetic samples become indistinguishable.
  • Black-Box (Disjoint-CNN): Achieves an average OOD F1 of 79.2, representing a 22.1% relative improvement over the white-box approach. It maintains a TPR@1%FPR of 57.2.
• Impact of Quality and Length: Detection becomes harder as synthetic data quality improves (e.g., Diffusion-TS produces high-quality data with lower detectability). Additionally, while black-box detectors maintain strong ID performance at longer sequence lengths (128), their OOD generalization degrades, suggesting that cross-generator artifacts become harder to capture as sequence length increases.

Significance and Claims
The paper asserts that detecting diffusion-generated time series is not a direct transfer of the image domain problem. The reliance on a universal reconstruction prior is a fundamental mismatch for time series, where generators are distribution-specific. Consequently, the authors conclude that black-box detection is a viable and superior starting point for this domain, despite being less interpretable than reconstruction-based methods. The work highlights that while current black-box detectors are effective, the specific signals they capture and their generalization across different generator types and dataset shifts remain open areas for future research.