⚡ electrical engineering

Dimensional Coactivation for Representational Consistency in Frozen Vision Foundation Models

This paper introduces Dimensional Coactivation (DCA), a novel metric that measures intra-sample representational consistency in frozen vision foundation models by analyzing raw feature dimension coactivation without standard normalization, demonstrating its effectiveness in detecting deepfakes by identifying structural incoherencies in synthetic faces.

Original authors: Izaldein Al-Zyoud Abdulmotaleb El Saddik

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Izaldein Al-Zyoud Abdulmotaleb El Saddik

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, frozen robot that has never been taught to spot fake photos. It has just learned how to "see" the world by looking at millions of real pictures. Now, you show it a picture of a face.

Usually, we ask the robot: "Does this look like a real person?" But this paper asks a different, more subtle question: "Does the robot see this face as a single, coherent story, or does it see it as a bunch of mismatched puzzle pieces?"

Here is the breakdown of their discovery, using simple analogies.

1. The Problem: The "Frankenstein" Face

Deepfakes (fake videos) are getting scary good. They can make a fake eye look perfect, a fake nose look perfect, and a fake mouth look perfect. If you look at just the eye, it's real. If you look at just the mouth, it's real.

But in a real human face, the eye, nose, and mouth are connected by a hidden "glue" of identity. They belong to the same person. In a fake face, that glue is broken. The parts look real individually, but they don't "speak the same language" to each other.

2. The Tool: A "Dimensional Coactivation" (DCA) Fingerprint

The researchers invented a new way to measure this "glue." They call it Dimensional Coactivation (DCA).

Think of the robot's brain as having 1,024 different "light switches" (dimensions) that turn on when it sees something.

Old Way: Most tools look at the whole face and say, "This looks 80% like a real face." They average everything out.
The New Way (DCA): This tool looks at specific pairs of features (like the eyes and the mouth) and asks: "When the robot sees the eyes, which specific light switches turn on? And when it sees the mouth, do those exact same switches turn on?"

If it's a real face, the same switches light up for the eyes and the mouth because they belong to the same identity. If it's a fake, the switches light up randomly for the eyes and differently for the mouth. The "glue" is missing.

3. The Secret Sauce: "Constraint Liberation"

This is the most important part of the paper. The researchers found that to see this "glue," you have to stop treating the robot's brain like a normal math problem.

Usually, when comparing things, mathematicians do three things to make it "fair":

Centering: Subtracting the average (removing the "baseline").
Normalization: Making sure everything is the same size (squashing big numbers down).
Mixing: Looking at how every switch interacts with every other switch.

The paper argues that for this specific job, these "fairness" rules actually destroy the signal.

The Analogy: Imagine you are trying to hear a whisper in a room.
- Centering is like turning off the volume of the room to hear the whisper better.
- Normalization is like forcing the whisper to be the same volume as a shout.
- Mixing is like listening to the echo of the whisper bouncing off every wall.

The researchers found that for a frozen robot (one that isn't being retrained), the volume (magnitude) and the baseline (mean) of the light switches actually contain the secret code of the face's identity. If you "normalize" or "center" the data, you accidentally erase the very thing you are trying to find. They call this "Constraint Liberation"—letting the raw data speak without forcing it into a box.

4. The Results: The "Eyes-Mouth-Nose" Fingerprint

They tested this on a famous fake-face detector called DINOv3.

They took the Eyes, Mouth, and Nose of a face.
They measured how the "light switches" co-activated between these three parts.
They created a 3,072-dimensional fingerprint (a long list of numbers) that describes the relationship between these parts.

The Score:

When they tested this on a dataset of deepfakes (CelebDF-v2), it was 91% accurate at spotting fakes, even though the robot had never seen those specific fakes before.
The "Aha!" Moment: When they tried to use the "fairness" rules (centering/normalizing) again, the accuracy crashed to 46% (basically guessing). This proved that their "raw" method was the only thing that worked.

5. Why the Robot Matters

They also tried swapping the robot's brain.

DINOv3 (Self-taught): Worked great. It had a stable "coordinate system" where the light switches meant the same thing for the eyes and the mouth.
FaRL (Taught to label parts): Worked terribly (58% accuracy). This robot was trained to just say "That's an eye, that's a nose." It didn't learn the deep, stable connection between them.

This proves that the ability to spot the "broken glue" depends entirely on the robot having a stable internal map of the world, not just on being able to find the face parts.

Summary

The paper says: Don't just look at the parts; look at how the parts talk to each other inside the robot's brain.

If you strip away the usual math rules (normalization, centering) and look at the raw "light switches" that turn on for the eyes and mouth, you can see a hidden fingerprint. If that fingerprint is messy, the face is fake. If it's consistent, the face is real. And surprisingly, the "messy" math rules we usually use to clean up data actually hide this truth.

Technical Summary: Dimensional Coactivation for Representational Consistency in Frozen Vision Foundation Models

1. Problem Definition: Representational Consistency (RC)

The paper addresses a gap in how frozen vision foundation models are evaluated. While these models are known to extract features, the authors posit that they also impose a learned coordinate system to organize visual structure. The central problem is Representational Consistency (RC): determining whether a single frozen model represents a single input as internally coherent across its semantic subregions.

Unlike standard similarity measures (e.g., CKA, RSA) that compare representations across different models, layers, or distributions, RC is an intra-sample, intra-model, inter-region measurement. It asks whether semantic regions of the same physical identity (e.g., eyes, nose, and mouth of a face) remain coordinated within the model's feature space. The authors argue that synthetic images (deepfakes) may reproduce locally plausible regions but fail to maintain the representational structure linking them, creating an internal inconsistency detectable by a frozen model trained on real-world imagery.

2. Methodology: Dimensional Coactivation (DCA)

To measure RC, the authors introduce Dimensional Coactivation (DCA), a per-dimension operator designed to preserve the specific properties of a frozen coordinate system.

Core Mechanism:
Given two semantic regions $r_1$ and $r_2$ with sampled patch-token matrices $F_{r_1}, F_{r_2} \in \mathbb{R}^{K \times D}$ , DCA computes the coactivation of the same feature dimensions across these regions:
$\text{DCA}(F_{r_1}, F_{r_2})[d] = \frac{1}{K} \sum_{k=1}^{K} F_{r_1}[k, d] \cdot F_{r_2}[k, d]$
This results in a vector of size $D$ (the feature dimension), representing the diagonal of the cross-region patch-token product. For a face, the method concatenates DCA vectors for the eyes-mouth, eyes-nose, and mouth-nose pairs to form a 3,072-dimensional fingerprint (using $D=1024$ ).

Constraint Liberation:
A critical aspect of the methodology is the deliberate avoidance of standard similarity preprocessing steps, termed "constraint liberation." The authors argue that operations useful for cross-model comparison are mismatched for intra-sample measurement:

No Centering: Mean activation is preserved because it may encode identity coherence rather than bias.
No L2 Normalization: Magnitude is preserved because stronger dimensions may carry heavier identity loads.
No Full Gram Coupling: Off-diagonal cross-dimension interactions are discarded to avoid noise and preserve the diagonal structure where the frozen coordinate system is readable.

Pipeline:

Feature Extraction: A frozen DINOv3 ViT-L/16 backbone extracts intermediate block-18 patch tokens (avoiding final normalized tokens).
Region Assignment: A frozen FaRL parser assigns tokens to semantic regions (eyes, mouth, nose).
Fingerprint Generation: DCA vectors are computed for the "eyes-mouth-nose" (EMN) triple.
Validation: A lightweight linear probe (logistic regression) is trained only on the source dataset (FF++ c23) to evaluate the fingerprint's ability to detect incoherence in target datasets (CelebDF-v2, DFD) zero-shot.

3. Key Contributions

Definition of RC: The paper defines Representational Consistency as a distinct measurement axis for frozen foundation models, focusing on intra-sample coherence rather than inter-model similarity.
DCA Operator: It proposes DCA as a per-dimension instrument that measures cross-region coherence without collapsing the feature space into scalar summaries or applying standard normalization constraints.
Constraint Liberation: It demonstrates that removing centering, L2 normalization, and cross-dimension coupling is not merely a simplification but a necessity for preserving the coherence signal in a fixed coordinate system.
Backbone Dependency: It validates that the success of RC measurement depends on the stability of the backbone's per-dimension coordinate system (e.g., DINOv3) rather than the region extraction pipeline alone.

4. Experimental Results

The method was validated using cross-dataset deepfake detection, where models trained on FF++ c23 were evaluated zero-shot on CelebDF-v2 and DFD.

Performance: The EMN DCA fingerprint achieved 0.9106 AUC on CelebDF-v2 and 0.9289 AUC on DFD.
Ablation of Constraints: Reintroducing standard constraints severely degraded performance:
- Adding centering dropped CelebDF-v2 AUC to 0.459.
- Adding L2 normalization dropped it to 0.862.
- Adding full Gram coupling (cross-dimension) dropped it to 0.478.
Backbone Sensitivity: Replacing DINOv3 with FaRL (a supervised parsing model) collapsed the CelebDF-v2 AUC to 0.582, indicating that FaRL lacks the stable per-dimension coordinate system required for RC.
Region Selection: The "eyes-mouth-nose" triple was found to be optimal. Including hair (spatially variable) or skin (diluting signal) reduced performance.
Scalar vs. Vector: Reducing the DCA fingerprint to scalar summaries (e.g., mean DCA, cosine similarity) resulted in AUCs between 0.47 and 0.66, proving that the discriminative signal is distributed across feature dimensions and lost in aggregation.

5. Significance and Claims

The paper positions DCA not as a universal deepfake detector superior to all existing methods, but as a measurement principle that reveals a specific type of signal: relational, feature-dimensional, and backbone-dependent coherence.

Measurement over Detection: The primary contribution is the demonstration that frozen foundation models contain measurable "coherence residuals" that expose deepfakes without fine-tuning. The signal is relational (how regions interact) rather than regional (how a single region looks).
Coordinate System Stability: The results suggest that models like DINOv3, trained with self-distillation, develop a stable coordinate system where the same semantic factors activate the same dimensions across spatial positions. This property is essential for RC measurement and is absent in models optimized for categorical boundaries (like FaRL).
Limitations: The authors modestly note that the method is currently face-specific, relies on a specific block depth (block 18) which was empirically selected, and has not been validated against modern diffusion-based manipulations or demographic robustness. The paper frames the work as a foundational measurement tool rather than a deployable forensic model.