Hyperbolic Latent Geometry for Tree-Structured Prototype Networks: A Local-vs-Global Trade-off

Technical Summary: Hyperbolic Latent Geometry for Tree-Structured Prototype Networks

Problem Statement

The paper investigates the efficacy of latent manifold geometry in hierarchical classification tasks. Specifically, it addresses the tension between satisfying a tree-structured regularizer (which enforces that class prototypes respect a known hierarchy) and maintaining high data likelihood (accurate classification).

Real-world classification problems often possess external hierarchies (e.g., artistic movements, biological taxa). A natural modeling approach is to add a regularizer that pulls the pairwise distances of class prototypes toward the distances defined by a reference tree. The core question is whether the choice of latent manifold—Euclidean space ($\mathbb{R}^d$) versus the Poincaré ball ($\mathbb{B}^d_c$)—affects the ability to satisfy this regularizer without distorting the class-conditional likelihood. Theoretical work suggests hyperbolic space, with its exponential volume growth, should embed trees with lower distortion than Euclidean space, which grows polynomially. However, it remains an open empirical question whether this geometric advantage translates to performance gains on real-world datasets when balancing likelihood and regularization.

Methodology

The authors instantiate this investigation using WikiArt-Refined, a dataset of 81,446 paintings labeled with 27 artistic styles. The hierarchy is hand-built from art-history references (e.g., Renaissance $\to$ Baroque $\to$ Rococo).

Model Architecture:

• Encoder: A frozen CLIP ViT-B/16 encoder extracts features.
• Head: A two-layer MLP maps features to a latent manifold $\mathcal{M}$.
• Classifier: A prototype classifier where class-conditional likelihood is a softmax over the distance to learnable class prototypes $\{p_k\} \subset \mathcal{M}$.
• Geometries: The study compares Euclidean $\mathbb{R}^d$ and Poincaré ball $\mathbb{B}^d_c$ (curvature $c > 0$). The only architectural differences are the final transform (identity vs. exponential map) and the distance metric (Euclidean vs. hyperbolic).

Regularization:
A tree-structured regularizer penalizes the deviation between the matrix of learned prototype distances ($D_{ij}$) and the reference tree distance matrix ($T_{ij}$). The regularizer is a mean-normalized squared error:
$$R(\{p_k\}) = \frac{\lambda}{|P|} \sum_{(i,j) \in P} \left( \frac{D_{ij}}{\bar{D}} - \frac{T_{ij}}{\bar{T}} \right)^2$$
where $\lambda$ controls the strength. This formulation ensures the regularizer constrains the relative structure of the prototypes, allowing the geometry to choose the absolute scale preferred by the likelihood.

Experimental Design:
The authors perform a sweep of 150 seed-replicated configurations varying:

• Embedding dimension $d \in \{2, 4, 8, 16, 32, 64\}$.
• Curvature $c \in \{0.1, 0.3, 1, 3\}$.
• Regularizer strength $\lambda \in \{0, 0.1, 0.3, 1, 3\}$.
• Three reference tree definitions: a hand-built lineage tree, a chronological era grouping, and a flat null tree.
• Two additional empirical trees derived from CLIP and DINOv2 feature centroids for robustness testing.

Key Results

The experiments yield three primary findings, distinguishing between local hierarchy preservation, global fidelity, and classification performance.

1. Local Hierarchy Preservation (The Robust Effect)
Hyperbolic prototypes substantially outperform matched Euclidean prototypes in preserving local tree structure.

• Metrics: Sibling recall@5 and cousin recall@5 (measuring if nearest neighbors in latent space are true siblings/cousins in the tree).
• Performance: At $d=8$, hyperbolic prototypes achieve a sibling recall@5 of 0.195, compared to 0.149 for Euclidean.
• Aggregated Gap: Across 18 paired seed configurations, the mean improvement is +8.7 percentage points (pp) for sibling recall and +15.2 pp for cousin recall.
• Robustness: This advantage persists across all embedding dimensions, regularizer strengths, and crucially, across three different reference tree definitions (hand-built, CLIP-derived, and DINOv2-derived). The gap widens as dimension increases for Euclidean (due to distortion) but remains flat for Hyperbolic.

2. Calibration and Classification (The Neutral/Trade-off Effect)
When compared to baselines trained on raw frozen CLIP features:

• Euclidean: Matches the performance of a logistic regression baseline on top-1 accuracy (64.3% vs. 64.1%) but fails to improve upon a k-NN baseline for local retrieval. It adds no detectable structural value over the encoder.
• Hyperbolic: Underperforms on top-1 classification (60.6%) compared to the Euclidean fit and the logistic baseline. However, it is the only trained model that improves upon the k-NN baseline for local hierarchy preservation (sibling recall@5 of 0.188 vs. 0.160 for k-NN).
• Conclusion: The hyperbolic fit trades global classification accuracy for local structural fidelity, whereas the Euclidean fit offers no structural advantage over the raw encoder.

3. Global Tree Fidelity (The Unstable Effect)
Claims regarding global tree fidelity are inconclusive and highly sensitive to the reference tree definition.

• Distortion: Mean tree distortion favors Euclidean space significantly against the hand-built tree.
• Correlation: The Spearman correlation between learned and tree distances is not significantly different from zero across the full sweep ($p=0.68$).
• Instability: The direction of the global fidelity gap flips depending on whether the reference tree is hand-built, CLIP-derived, or DINOv2-derived. The authors explicitly state they do not claim a "winner" for global tree fidelity.

Significance and Claims

The paper's primary contribution is an empirical separation between Euclidean and hyperbolic latent geometries for a class-structured regularizer on a real hierarchical classification problem.

• Local vs. Global Trade-off: The authors claim a "clean" result: hyperbolic latent structure adds local-retrieval value over the encoder (improving sibling/cousin recall), while Euclidean structure does not. This local advantage is robust to the definition of "local" (i.e., the choice of reference tree).
• No Global Winner: The paper modestly refrains from claiming that hyperbolic geometry is universally superior for global tree embedding, noting that global fidelity metrics are unstable and dependent on the reference tree construction.
• Mechanism: The results support the theoretical intuition that hyperbolic space's exponential volume growth allows it to satisfy tree-structured constraints "cheaper" (with less distortion to the likelihood) than Euclidean space, but this manifests primarily in the preservation of local neighborhood topology rather than global classification accuracy.

Limitations:
The authors note that the findings are specific to medium-scale datasets with frozen encoders (CLIP ViT-B/16) and a 27-leaf taxonomy. They do not claim the local advantage would necessarily survive a fine-tuned encoder or a significantly deeper hierarchy. Additionally, the classification gap suggests that while hyperbolic geometry improves structural retrieval, it does not necessarily improve the discriminative power of the classifier itself in this setup.