💻 computer science

ProDG: Prototypes for Data-Free Generative Post-Hoc Explainability

This paper introduces ProDG, a novel data-free framework that leverages generative models to synthesize high-fidelity visual prototypes directly from frozen model weights, thereby enabling robust post-hoc interpretability for privacy-sensitive domains without requiring access to any external data.

Original authors: Piotr Borycki, Magdalena Trędowicz, Jacek Tabor, Łukasz Struski, Przemysław Spurek

Published 2026-05-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Piotr Borycki, Magdalena Trędowicz, Jacek Tabor, Łukasz Struski, Przemysław Spurek

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart robot that can look at a picture and tell you exactly what it is—like identifying a specific breed of dog or a rare bird. But here's the catch: the robot is a "black box." It gives you the answer, but it won't tell you why it thinks that. It's like a chef who serves you a perfect meal but refuses to tell you what ingredients they used.

For a long time, scientists tried to peek inside the black box. Some methods just highlighted "hot spots" on the image (like a heat map), but those spots were often blurry and didn't make much sense to humans. Others tried to find real examples from the robot's training data that looked like the answer, saying, "See? This picture looks like that one, so that's why I chose it."

The Problem with the Old Way
The problem with finding real examples is that you need access to the robot's original training data. But in many real-world situations—like in hospitals with private patient records or banks with sensitive financial data—that data is locked away. You can't look at it. So, the old methods hit a wall: they can't explain the robot if they can't see the data.

The New Solution: ProDG
This paper introduces ProDG (Prototypes for Data-Free Generative Post-Hoc Explainability). Think of ProDG as a "magical sketch artist" that doesn't need to see the original photo album to understand the robot's mind.

Here is how it works, using simple analogies:

1. The "Brain Surgery" (Feature Disentanglement)

The robot's brain (its neural network) is messy. When it sees a "wolf," it might activate 50 different parts of its brain at once, all jumbled together.

The Fix: ProDG performs a gentle "brain surgery." It uses a special mathematical tool (called an Orthogonal Feature Disentanglement Module) to untangle the mess. It rearranges the robot's internal wiring so that each specific wire (or "channel") is dedicated to just one clear idea, like "wolf eyes" or "wolf snout," without the other ideas getting in the way.
The Magic: It does this without changing the robot's actual answers. It's like reorganizing a library so books are easier to find, but the books themselves and the librarian's knowledge remain exactly the same.

2. The "Dream Generator" (Generative Prototypes)

Usually, to explain the robot, you'd need to dig through a pile of real photos to find a perfect example of a "wolf snout." But since we can't see the real photos, ProDG uses a Generative Model (like a high-tech AI artist).

The Process: ProDG talks to this AI artist. It says, "Hey, I need you to draw a picture that makes the 'wolf snout' wire in the robot's brain light up as brightly as possible."
The Result: The AI artist doesn't copy a real photo; it dreams up a brand new, perfect "wolf snout" image from scratch. It keeps trying different sketches until it finds the one that makes the robot's brain say, "Yes! That is exactly what I was thinking!"

3. The "Concept Bank" (Prompt Optimization)

To make sure the AI artist draws the right thing, ProDG uses a Concept Prompt Bank. Imagine this as a dictionary of instructions.

Instead of just saying "draw a wolf," ProDG fine-tunes the instructions (the "prompts") to be incredibly specific. It tweaks the instructions slightly to ensure the artist draws a variety of different "wolf snouts" (some with fur, some with scars, some in different lighting) so the explanation isn't just one boring, static image. This prevents the artist from getting stuck drawing the exact same picture every time.

Why This Matters

Privacy First: Because ProDG creates these explanations from scratch using the robot's internal "weights" (its brain structure) and a generative artist, it never needs to see the original private data. You can explain a medical diagnosis without ever looking at the patient's private records.
Clearer Explanations: Instead of showing a blurry red blob on a picture (like older methods), ProDG shows you a clear, high-quality image of the specific feature the robot is looking at (e.g., "I see the antennae, so I know this is an insect").

The Bottom Line

ProDG is a new way to open the black box. It takes a frozen, unchangeable AI, untangles its messy brain, and then uses a generative AI artist to "dream up" perfect examples of what the robot is thinking. This allows us to understand complex AI decisions even when the original data is strictly off-limits, keeping privacy safe while making the AI's reasoning crystal clear.

Technical Summary: ProDG – Prototypes for Data-Free Generative Post-Hoc Explainability

Problem Statement

Deep neural networks have achieved remarkable performance in computer vision but remain "black boxes," raising critical trust and safety issues. While post-hoc explainability methods (e.g., Grad-CAM, LRP) offer efficient explanations for single images without retraining, they typically produce pixel-level heatmaps that lack clear semantic meaning, forcing users to infer the model's reasoning. Conversely, ante-hoc prototype-based methods (e.g., ProtoPNet) provide human-understandable "this looks like that" reasoning but require specialized architectures and extensive retraining.

Recent hybrid approaches, such as EPIC, attempt to bridge this gap by extracting prototype-like explanations from frozen classifiers without retraining. However, these methods face a fundamental limitation: they require access to a subset of real-world data (e.g., a validation or test set) to search for, crop, and extract visual prototypes. This dependency renders them unusable in privacy-sensitive domains (e.g., healthcare, finance) where original datasets are strictly restricted or inaccessible.

Methodology: ProDG Framework

ProDG (Generative Prototypes for Data-Free Post-Hoc Explainability) addresses the data dependency issue by synthesizing pure, high-fidelity visual prototypes directly from the weights of a frozen classifier, eliminating the need for any external data. The framework consists of three primary components:

1. Orthogonal Feature Disentanglement Module

In standard networks, visual features are often entangled across multiple channels. ProDG introduces an orthogonal change of basis matrix $U \in \mathbb{R}^{C \times C}$ to map the original entangled activations $\Phi(x)$ into a new space $Z = U \Phi(x)$ .

Goal: Align new channel representations with distinct, semantically meaningful visual concepts so that each channel activates strongly for exactly one specific feature.
Faithfulness Guarantee: Since $U$ is constrained to be orthogonal, its inverse is simply its transpose ( $U^{-1} = U^T$ ). By modifying the linear classification head to incorporate this inverse mapping (multiplying classifier weights by $U^T$ ), the change of basis is mathematically canceled out during the forward pass. This ensures the modified network retains the exact predictive accuracy and logit predictions of the original base model.

2. Concept Prompt Bank

To generate prototypes without real data, ProDG leverages a frozen generative model (specifically FLUX.1-schnell) conditioned on text embeddings.

Parameterization: A "Concept Prompt Bank" $\mathcal{P} = \{P_c\}_{c=1}^C$ is defined for each channel. Each $P_c$ includes frozen anchor embeddings (derived from class names during an initial discovery phase) and learnable distributional parameters.
Optimization: The learnable parameters include offsets and variances for the text embeddings. For high-dimensional T5 embeddings, Low-Rank Adaptation (LoRA) is used; for CLIP embeddings, direct offsets are learned.
Diversity: Embeddings are formulated as probabilistic distributions. Using the reparameterization trick, the system samples from these distributions to generate diverse prototype images rather than a single static image per concept.

3. Alternating Optimization Scheme

The framework jointly optimizes the transformation matrix $U$ and the prompt bank parameters $\Theta_c$ using an alternating strategy:

Phase 1 (Update $U$ ): Freeze prompt parameters, sample images for target channels, and update $U$ to maximize the purity metric (the ratio of the maximum activation in a specific channel to the norm of the feature vector at that spatial location).
Phase 2 (Update Prompts): Freeze $U$ $U$ , generate images, and update embeddings to maximize purity while applying two regularization losses:
- Regularization Loss ( $L_{reg}$ ): Penalizes large deviations of learned embeddings from meaningful natural language anchors to maintain semantic alignment.
- Diversity Loss ( $L_{div}$ ): Encourages the generation of different images for the same channel to prevent mode collapse, computed via pairwise cosine similarity of feature vectors.

Explanation Generation

During inference, the system identifies the top- $k$ most influential channels for a predicted class. It then samples from the optimized prompt bank to generate corresponding images. Visual explanations are derived by computing activation heatmaps based on the product of Spatial Purity (dominance of the channel) and Relative Magnitude (normalized activation), resulting in localized bounding boxes that highlight prototypical regions.

Key Contributions

Data-Free Framework: ProDG is the first prototype-based method that synthesizes prototype explanations without requiring access to original or validation datasets.
Purity-Optimized Conditioning: The introduction of a mechanism that utilizes generative models to create high-fidelity, interpretable visual concepts directly from the weights of a frozen classifier.
Privacy-Preserving Interpretability: The method provides a secure alternative to traditional data-dependent extraction, offering cleaner conceptual representations free from irrelevant background noise, suitable for restricted environments.

Results and Evaluation

The authors evaluated ProDG on four benchmarks (CUB-200-2011, Stanford Cars, Stanford Dogs, and ImageNet) using various backbones (ResNet, DenseNet, ConvNeXt, SwinTransformer).

Predictive Faithfulness: As theoretically guaranteed, ProDG maintains the baseline model's accuracy exactly (0% difference) across all datasets and architectures, matching the performance of EPIC and InfoDisent.
Qualitative Performance: Visual comparisons show ProDG produces semantically rich cues (e.g., object structure, texture) comparable to data-dependent methods like EPIC, capturing discriminative regions effectively.
Intra-Concept Diversity: Using the Learned Perceptual Image Patch Similarity (LPIPS) metric, ProDG demonstrated high scores ( $\geq 0.61$ ), indicating the framework successfully avoids mode collapse and generates diverse visual variations for the same concept.
User Study: In a study on ImageNet, participants achieved significantly higher-than-chance accuracy (73% vs. random selection) when identifying classes based on ProDG explanations. In a three-way comparison (ProDG vs. InfoDisent vs. EPIC), ProDG achieved the highest average rank (1.85), with statistically significant differences found against InfoDisent.

Significance and Limitations

The paper claims that ProDG establishes a new frontier in Data-Free XAI, unlocking robust visual interpretability for highly constrained environments where data access is impossible. By decoupling prototype discovery from data availability, it enables transparency in privacy-sensitive domains.

The authors modestly acknowledge limitations inherent to the generative approach:

Generative Prior Bias: If target concepts are entirely out-of-distribution for the underlying generative model (e.g., FLUX), the prompt optimization may struggle to synthesize valid prototypes.
Domain Adaptation: Applying the framework to specialized domains may require swapping the generative prior for a domain-specific model.
Initialization: While text anchor initialization aids convergence, more advanced initialization strategies could potentially improve the capture of highly abstract visual features.