🤖 machine learning

Hyperbolic Concept Bottleneck Models

The paper introduces Hyperbolic Concept Bottleneck Models (HypCBM), a post-hoc framework that leverages hyperbolic geometry to model concept hierarchies through asymmetric containment, achieving superior interpretability, hierarchical consistency, and robustness compared to Euclidean models without requiring additional supervision or pre-training penalties.

Original authors: Daniel Uyterlinde, Swasti Shreya Mishra, Pascal Mettes

Published 2026-05-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Daniel Uyterlinde, Swasti Shreya Mishra, Pascal Mettes

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer to recognize a penguin.

In the old way of doing this (called a standard "Concept Bottleneck Model"), the computer is given a flat list of features, like a spreadsheet. It sees "has wings," "has feathers," "lives in water," and "is a bird." In this flat list, "bird" and "penguin" are just two separate boxes sitting side-by-side. The computer doesn't inherently know that "penguin" is a type of "bird." If you tell the computer, "This is not a bird," it might still think, "Okay, but it's definitely a penguin," because in that flat list, the two ideas aren't connected. This leads to logical mistakes.

Hyperbolic Concept Bottleneck Models (HypCBM) fixes this by changing the shape of the computer's "mind."

The Analogy: A Tree vs. A Flat Map

Imagine trying to organize a library.

The Old Way (Euclidean Space): You try to fit the entire library onto a single, flat floor. To fit everything, you have to squish the books together. A "Fiction" section and a "Science Fiction" section end up overlapping or getting distorted. It's hard to see that "Science Fiction" is a small branch growing out of the big "Fiction" tree.
The New Way (Hyperbolic Space): Imagine the library is shaped like a giant, expanding tree or a funnel. The top of the funnel is the broad, general ideas (like "Animal"). As you go deeper into the funnel, the space expands exponentially, allowing you to fit specific, detailed ideas (like "Penguin," "Emperor Penguin," "Blue-footed Booby") without them bumping into each other.

In this new shape, the computer naturally understands that "Penguin" lives inside the "Bird" section. It's not just a neighbor; it's a child of the parent concept.

How It Works (The "Entailment Cone")

The paper introduces a clever trick called Entailment Cones.

Think of a concept like "Bird" as a flashlight beam shining down from the top of the funnel.

The beam gets wider as it goes down.
If an image (like a photo of a penguin) falls inside the "Bird" beam, the computer knows, "Yes, this is a bird."
If the image falls inside the "Bird" beam but also inside a narrower, more specific beam called "Penguin," the computer knows, "This is a penguin."

The magic is that the computer doesn't need to be explicitly taught the rules of "Birds contain Penguins." The shape of the space (the funnel) forces this logic to happen automatically. If you turn off the "Bird" beam, the "Penguin" beam automatically turns off too, because the penguin is trapped inside the bird's light.

Why This Matters

The authors tested this on three main things:

It's Smarter with Less Data:
Imagine two students taking a test. Student A (the old model) studied with a massive textbook (400 million examples). Student B (HypCBM) studied with a much smaller book (20 million examples). Usually, Student A wins. But because Student B's study method (the funnel shape) is so much better at organizing information, Student B actually beat Student A, even with 20 times less data.
It Makes Fewer Logical Mistakes:
If you tell the old model, "This is not a dog," it might still say, "But it's a Golden Retriever!" (a logical contradiction). The new model understands the hierarchy. If you say "Not a dog," it automatically knows it can't be a Golden Retriever either. It keeps its explanations consistent.
It's More Robust:
If you take a photo and add some static noise or blur it (like a bad internet connection), the old model gets confused and changes its mind about what concepts are present. The new model stays steady. It's like looking at a tree in the wind; the leaves (the specific details) might shake, but the trunk (the main structure) stays solid.

The "Human-in-the-Loop" Superpower

The best part is how easy it is to fix the computer's mistakes.
If the computer thinks a picture of a locker room is a server room (because both have "electrical equipment"), you can simply tell the computer, "No, this is not electrical equipment."

Old Model: You have to manually check every single box related to "server room" to fix it.
New Model: You just turn off "electrical equipment." Because of the funnel shape, the computer automatically turns off "server room," "circuit breaker," and "technical equipment" for you. It fixes the whole branch of the tree with one click.

Summary

The paper claims that by moving from a flat, messy list of ideas to a structured, tree-like shape (hyperbolic space), we can make AI models that:

Understand how concepts relate to each other (parents and children).
Learn faster and with less data.
Make fewer logical errors.
Are easier for humans to correct when they get things wrong.

The authors call this HypCBM, and they show it works better than previous methods, even when those methods were trained on much larger amounts of data.

Technical Summary: Hyperbolic Concept Bottleneck Models (HypCBM)

1. Problem Statement

Concept Bottleneck Models (CBMs) have emerged as a leading approach for interpretable machine learning, forcing models to predict human-understandable concepts before making a final decision. This architecture enables transparency and allows for human-in-the-loop interventions to correct erroneous predictions. However, existing CBMs rely on a fundamental representational flaw: they embed concepts in flat, Euclidean space, treating them as independent, orthogonal dimensions.

In reality, concepts are highly structured and organized in semantic hierarchies (e.g., "Dog" is a subset of "Animal"). Euclidean geometry struggles to represent these branching relationships without forcing parent and child concepts into overlapping or distorted regions, leading to logical inconsistencies (e.g., predicting "Dog" without "Animal") and inefficient interventions. Furthermore, current methods that attempt to enforce structure often require additional supervised training or learned modules, deviating from the desirable "post-hoc" paradigm where the model is interpretable without retraining the backbone.

2. Methodology

The authors propose Hyperbolic Concept Bottleneck Models (HypCBM), a post-hoc framework that grounds the concept bottleneck in hyperbolic geometry to naturally accommodate semantic hierarchies.

2.1 Geometric Foundation

HypCBM utilizes the Lorentz model of hyperbolic space, a Riemannian manifold of constant negative curvature. Unlike Euclidean space, hyperbolic space exhibits exponential volume growth, making it the natural domain for hierarchical representation learning. The framework leverages pre-trained hyperbolic Vision-Language Models (VLMs), specifically HyCoCLIP, which aligns image and text embeddings in this space using entailment cone objectives.

2.2 Concept Generation and Filtering

Following a label-free, post-hoc paradigm, HypCBM generates a candidate set of concepts using a Large Language Model (LLM) prompted with dataset class labels. These textual descriptions are encoded into the hyperbolic manifold using the frozen text encoder of the VLM.

Norm-Based Filtering: To remove overly generic concepts (e.g., "object") that lack discriminative power, the framework filters concepts based on their distance from the origin in hyperbolic space. Concepts with low norms (near the origin) represent broad, generic categories and are pruned, as they tend to activate indiscriminately.

2.3 Hyperbolic Entailment Activation

The core innovation is the reformulation of concept activation from a symmetric distance metric (like cosine similarity) to an asymmetric geometric containment problem.

Entailment Cones: Each concept $c_i$ is associated with an entailment cone defined by a half-aperture $\omega(c_i)$ , which is inversely proportional to the concept's spatial norm (specific concepts have narrower cones).
Activation Mechanism: An image embedding $z$ activates a concept if it falls geometrically within the concept's entailment cone. The activation value $a_i$ is calculated as the "margin of inclusion"—the degree to which the image embedding is contained within the cone, normalized by the cone's aperture.
Result: This yields a sparse, non-negative activation vector that is hierarchically consistent by construction. If an image entails a specific child concept, it geometrically entails the parent concept.

2.4 Hierarchically Faithful Interventions

A critical limitation of Euclidean CBMs is that intervening on a parent concept (e.g., setting "Animal" to 0) does not automatically deactivate child concepts (e.g., "Dog"), leading to logical paradoxes. HypCBM resolves this through adaptive scaling:

Due to the exponential volume growth of hyperbolic space, a fixed threshold for entailment is insufficient across different hierarchy depths.
The authors derive an adaptive scaling law for the entailment strictness parameter $\eta_{text}$ , which scales linearly with the parent concept's norm.
When a user suppresses a parent concept, the framework automatically propagates this suppression to all geometric descendants satisfying the adaptive entailment condition, ensuring coherent corrections throughout the concept tree.

3. Key Contributions

Hyperbolic Entailment Activation: A mechanism that maps image representations to sparse, hierarchy-aware concept activations via entailment cone containment, eliminating the need for symmetric distance metrics.
Adaptive Scaling Law: A novel strategy for hierarchical interventions that propagates user corrections coherently through the concept tree by adapting strictness to the geometric specificity of parent concepts.
Post-Hoc Structured Bottleneck: HypCBM is the first framework to achieve a structured bottleneck completely post-hoc without learned modules or additional supervision, relying solely on the geometric properties of pre-trained hyperbolic VLMs.

4. Experimental Results

The authors evaluated HypCBM on CIFAR-100, SUN397, and ImageNet, comparing it against Euclidean baselines (LF-CBM) and models trained on significantly more data.

Accuracy and Data Efficiency: HypCBM consistently outperforms Euclidean post-hoc models. Notably, it rivals Euclidean models trained on 20× more data in sparse regimes (low concept budgets). For example, on ImageNet with a budget of 10 concepts, HypCBM more than doubles the baseline accuracy (34.6% vs. 15.3%).
Hierarchical Consistency: HypCBM demonstrates significantly higher logical consistency. In strict budget regimes, it achieves over 100% relative improvement in consistency compared to Euclidean baselines, ensuring that activating a child concept implies the activation of its parent.
Intervention Responsiveness: In human-in-the-loop simulations, HypCBM shows steeper confidence decay when correcting errors, successfully flipping incorrect predictions to the correct class for 19% more samples than the Euclidean baseline.
Robustness: HypCBM exhibits superior robustness to input corruptions (noise, blur, weather). It maintains significantly higher "concept stability" (semantic consistency of explanations) under perturbation compared to Euclidean models, which suffer from rapid degradation in explanation quality.

5. Significance and Claims

The paper claims that structural priors on the representation space can serve as a powerful substitute for scale in sparse, interpretable regimes. By embedding concepts in a manifold that reflects their hierarchical relationships, HypCBM produces explanations that are more accurate, robust, and logically consistent without requiring additional training or supervision.

The authors emphasize that interpretability is not merely a property of model outputs but is fundamentally shaped by the geometry of the representation space. HypCBM demonstrates that leveraging the natural exponential volume growth of hyperbolic space allows for efficient disentanglement of semantic information, enabling models to achieve high performance and interpretability with less data than their Euclidean counterparts. The framework is presented as a zero-shot pathway to interpretability for the emerging class of hyperbolic foundation models.