Technical Summary: Amplifying Membership Signal Through Chained Regeneration

Problem Statement

The rapid scaling of generative AI has intensified the risk of training data memorization, creating urgent needs for privacy auditing and copyright enforcement. Existing methods for Membership Inference Attacks (MIAs) and Dataset Inference (DI) face significant limitations:

• Weak Signals: Most current approaches rely on "one-shot" generations (a single query), which often yield fragile signals that degrade under distributional shifts, performing only marginally better than random guessing.
• Computational Infeasibility: High-performance "shadow model" attacks, which train auxiliary models to simulate the target, are computationally prohibitive for large-scale generative architectures.
• Modality Limitations: Many existing techniques are not easily transferable across diverse modalities (vision, language, audio) or model families.

The core challenge is to detect whether a specific sample or dataset was used to train a model without relying on expensive shadow training or fragile single-query signals.

Methodology: MADreMIA

The authors propose MADreMIA, a model-agnostic framework that shifts the paradigm from static, single-pass inference to dynamic, iterative trajectories. Inspired by Model Autophagy Disorder (MAD)—a phenomenon where models trained on their own synthetic outputs eventually collapse—the authors pivot this concept into a diagnostic tool.

Core Mechanism

Instead of a single query, MADreMIA constructs an iterative regeneration chain $Z_0, Z_1, \dots, Z_T$:

1. Input: A sample $Z_0$ is fed into the target model $f$.
2. Regeneration: The output $Z_1$ is fed back as the input for the next step ($Z_{t+1} = R(f, Z_t)$), creating a recursive loop.
3. Trajectory Analysis: The framework analyzes the evolution of the sample over $T$ steps.

Theoretical Hypothesis

The method relies on a fundamental asymmetry in how models process data:

• Re-members (Training Data): Because the model has "memorized" these points, they act as stable attractors in the latent space. When iteratively regenerated, they exhibit high coherence and slow semantic degradation.
• Re-non-members (Unseen Data): Lacking a structural anchor in the model's weights, these samples drift rapidly toward the model's average biases or dissolve into noise.

Technical Implementation

• Modality-Agnostic: The framework adapts the regeneration operator $R$ to specific modalities:
  • Images (IARs & Diffusion): Uses fixed controls for autoregressive decoding or partial forward noising/re-denoising. Features include MSE, LPIPS, and SSIM relative to $Z_0$.
  • Language (LLMs): Uses an autophagous text loop with fixed context windows. Features include Kullback-Leibler Divergence (KLD), Jensen-Shannon Divergence, Jaccard Index, and Logit Margins.
  • Audio: Uses iterative reconstruction loops, measured via Fréchet Audio Distance (FAD).
• Signal Fusion: The system aggregates trajectory statistics (drift, consistency, quality evolution) into a fused representation $\tilde{z}$, which is fed into a calibrated scorer (typically L1-regularized logistic regression).
• Threat Model: The framework operates as a black-box, gray-box, or white-box extension, requiring no access to training data identities and no shadow model training.

Key Contributions

1. Iterative Regeneration Setup: Introduction of a method to uncover data memorization that is invisible during single-pass inference by probing semantic and structural consistency under repeated self-interaction.
2. Theoretical and Empirical Validation: Demonstration that trajectory features (generation dynamics over time) yield a significantly more statistically robust membership signal than one-shot baselines. The authors prove theoretically that trajectory averaging acts as a variance reduction mechanism, isolating membership information more effectively.
3. Cross-Modal Framework: A unified approach that improves MIA and DI efficiency across Vision (IARs, Diffusion), Language (LLMs), and Audio models without the computational cost of shadow model training.

Experimental Results

The authors evaluated MADreMIA across diverse model families and modalities using public datasets (ImageNet, COCO, and various LLM benchmarks).

• Qualitative Divergence: Visual and quantitative analysis (Figure 2 and Figure 3) confirms that "re-members" maintain structural quality and semantic coherence significantly longer than "re-non-members," which degrade rapidly.
• Performance Gains:
  • LLMs: Augmenting baseline attacks (e.g., Loss, Zlib, Min-K%, CAMIA) with trajectory features consistently improved performance. For OLMo-7B, the Zlib baseline (AUC 0.179) recovered to 0.868 with combined signals, and CAMIA reached 0.969.
  • Image Models: On IARs, MADreMIA improved classification accuracy significantly (e.g., VAR-d30 from 0.607 to 0.696; RAR-XXL from 0.562 to 0.713).
  • Dataset Inference (DI): The method strengthened statistical evidence for dataset-level inference, shifting p-value distributions rightward and allowing detection with fewer samples compared to baselines.
• Robustness: The signal remained robust across different regeneration strengths and model sizes, though the separation between members and non-members converged as regeneration became more aggressive.

Significance and Claims

The paper claims that MADreMIA provides a practical, scalable, and model-agnostic solution for privacy auditing. By leveraging the inherent "truth" of the training set as a stable attractor, the method amplifies weak membership signals that are otherwise lost in noise.

The authors emphasize that this approach does not increase the theoretical Bayes information ceiling but improves practical fixed-form statistics via temporal variance reduction. This makes it a viable tool for real-world auditing of large generative models where shadow model training is infeasible, offering a powerful signal amplifier for detecting training data membership across vision, language, and audio modalities.