A Fiber Criterion for Representation Identifiability in Supervised Learning

Technical Summary: A Fiber Criterion for Representation Identifiability in Supervised Learning

Problem Statement
In supervised learning, models are typically evaluated based on their input-output behavior (predictive accuracy or risk). However, modern models are often interpreted not merely as predictors but as representation learners, where the internal mapping $h$ is claimed to possess specific structural properties such as compression, invariance, disentanglement, or semantic accessibility. The paper identifies a fundamental gap: supervised evidence constrains the composite predictor $f = c \circ h$ (where $h$ maps inputs to representations and $c$ maps representations to outputs), but it does not necessarily determine the specific factorization $(h, c)$ that realizes it.

Two distinct factorizations can induce the exact same predictor $f$ while possessing substantially different internal representations. For instance, one representation might discard nuisance information while another retains it; one might be compressed while another is not. The paper formalizes the question: Which properties of the internal representation $h$ are uniquely determined by the induced predictor $c \circ h$? The authors argue that without additional assumptions, many common representation-level claims are not identifiable from supervised predictive behavior alone.

Methodology and Theoretical Framework
The paper approaches this problem through a formal identifiability framework based on the concept of fibers in the space of representation-head pairs.

1. Predictor Equivalence and Fibers:
   Let $\mathcal{M}$ be a class of admissible representation-head pairs $(h, c)$. The paper defines a projection map $\Pi: \mathcal{M} \to \Pi(\mathcal{M})$ where $\Pi(h, c) = [c \circ h]_{P_X}$, representing the induced predictor identified up to $P_X$-almost-sure equality.

   • Two pairs $(h, c)$ and $(\tilde{h}, \tilde{c})$ are predictor-equivalent if they induce the same predictor ($c \circ h = \tilde{c} \circ \tilde{h}$ almost surely).
   • The set of all pairs inducing the same predictor forms a fiber of $\Pi$.

2. The Fiber/Descent Criterion:
   The central theoretical result (Theorem 3.3) establishes a necessary and sufficient condition for representation identifiability:

   • A representation property $P$ is identifiable from the induced predictor if and only if $P$ is constant on every fiber of $\Pi$.
   • Equivalently, $P$ must "descend" to a well-defined property of the predictor itself. If $P$ varies within a single fiber (i.e., two factorizations produce the same predictor but have different values for $P$), then $P$ cannot be inferred from the predictor alone.

3. The Augmentation Obstruction:
   To demonstrate non-identifiability, the paper introduces a canonical construction called predictor-preserving augmentation.

   • Given a factorization $(h, c)$ and an auxiliary statistic $q(x)$, one can construct a new representation $h_q(x) = (h(x), q(x))$ and a new head $c_q(u, v) = c(u)$.
   • The new head ignores the appended coordinate $q$, ensuring $c_q \circ h_q = c \circ h$ pointwise. The predictor and supervised risk remain unchanged.
   • However, the new representation $h_q$ may differ from $h$ in properties such as minimality, compression, invariance, or semantic accessibility.
   • Theorem 3.5 (Augmentation Obstruction): If a model class $\mathcal{M}$ is closed under predictor-preserving augmentation, any representation property that changes when such augmentation is applied is not identifiable from the induced predictor (and consequently not from scalar supervised risk).

4. Restoring Identifiability:
   The paper notes that identifiability can be restored by restricting the admissible class $\mathcal{M}$ (e.g., via architectural constraints, bottlenecks, or inductive biases) such that the "bad" augmentations are excluded, or by enriching the observational object (e.g., adding auxiliary losses, multiple environments, or direct measurements).

Key Contributions

• Formalization of Representation Identifiability: The paper defines representation-level identifiability strictly in terms of the induced predictor, distinguishing it from statistical identifiability of latent generative factors.
• The Fiber/Descent Criterion: It provides a rigorous mathematical criterion (Theorem 3.3) stating that a property is identifiable iff it is constant on the fibers of the projection from factorizations to predictors.
• General Obstruction Mechanism: It identifies predictor-preserving augmentation as a universal witness for non-identifiability, applying to common desiderata including minimality, compression, invariance, equivariance, nuisance removal, and semantic accessibility (Table 1).
• Empirical Diagnostics: The paper provides two types of empirical evidence:
  1. Exact Algebraic Witnesses: Experiments on CelebA, CIFAR-10, STL-10, OfficeHome, and PACS where auxiliary information (semantic attributes, transformation embeddings, domain labels) is appended to representations. The results show that while the predictor remains identical (zero disagreement), standard diagnostics (probe accuracy, invariance distance, effective rank) change drastically.
  2. Matched-Performance Study: A study on the Waterbirds dataset comparing models trained with different constraints (ERM, Bottleneck, VIB, AugInv, SupCon) that achieve similar supervised performance. The results show that these models select different representation diagnostics (e.g., effective rank, nuisance decodability) despite having comparable test accuracy and loss.

Results

• Theoretical: The paper proves that properties like compression, invariance, and semantic accessibility are not consequences of supervised prediction alone. They vary within the fiber of the induced predictor unless the admissible class is restricted.
• Empirical:
  • In the exact witness experiments, appending a single coordinate (e.g., a domain label) to a representation allowed a probe to decode that information with 100% accuracy, whereas the original representation could not, despite the predictor's output being unchanged.
  • In the Waterbirds study, models with matched test accuracy exhibited significant divergence in representation properties. For example, Information Bottleneck methods selected extremely low-rank representations (effective rank $\approx 1.8$) compared to ERM (effective rank $\approx 35.7$), while augmentation-invariance methods reduced transformation sensitivity but maintained high nuisance decodability. Crucially, worst-group accuracy did not monotonically correlate with nuisance probe accuracy, indicating that representation-level claims require specific justifications beyond predictive performance.

Significance and Claims
The paper claims to clarify the logical relationship between supervised predictive behavior and internal representation properties. Its primary contribution is a negative result regarding the sufficiency of supervised learning for representation claims:

• Supervised predictive behavior certifies a representation-level property only when that property is determined by the induced predictor.
• Claims about compression, invariance, or semantic structure are not automatic consequences of low supervised loss.
• To justify such claims, one must rely on additional structure beyond the predictor, such as architectural restrictions, regularization, bottlenecks, data augmentation, auxiliary objectives, causal assumptions, or direct representation-level measurements.

The paper concludes that representation-level identifiability is conditional on the admissible class of factorizations. Without constraints that shrink the predictor fiber or select specific representatives, the internal structure of a supervised model remains under-specified by the input-output behavior alone.