📊 statistics

Classification Fields: Arbitrarily Fine Recursive Hierarchical Clustering From Few Examples

This paper introduces "classification fields," a framework for learning infinite-depth hierarchical cluster structures from finite examples by inferring local parent-to-child refinement rules, and proves that these rules can be effectively approximated by neural networks to generate deep, geometrically consistent hierarchies.

Original authors: Yicen Li, Ruiyang Hong, Anastasis Kratsios, Haitz Sáez de Ocáriz Borde, Paul D. McNicholas

Published 2026-05-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yicen Li, Ruiyang Hong, Anastasis Kratsios, Haitz Sáez de Ocáriz Borde, Paul D. McNicholas

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are looking at a family tree. Usually, when we do "clustering" (grouping things together), we just draw a tree for the specific people we have in front of us right now. If we have 100 photos, we make a tree with 100 leaves. That's it. The tree stops there.

But what if the world isn't just a fixed list of 100 photos? What if the world is like a fractal? Think of a fern leaf: you see the big leaf, then you zoom in and see smaller leaves, then even smaller ones, and theoretically, you could keep zooming in forever, and the pattern would keep repeating.

This paper asks a big question: If we only see the top few branches of a giant, infinite family tree, can we figure out the "rule" that generates the rest of the tree?

Here is the breakdown of their idea, using simple analogies:

1. The Problem: The "Finite" vs. The "Infinite"

Most computer programs for grouping data are like a photographer taking a picture of a crowd and drawing lines to group them. Once the photo is taken, the job is done. They don't know how to imagine the crowd if 1,000 more people showed up.

The authors say: "Wait, what if the grouping rule is actually a recipe?"
Instead of just memorizing the 100 people we see, we want to learn the instruction manual that tells us how to create the next level of groups, and the level after that, forever.

2. The Solution: "Classification Fields"

They invented a new concept called a Classification Field.

The Analogy: Imagine a magical stamp. You press it on a piece of paper (a "parent" group), and it doesn't just make a copy; it creates a specific pattern of three new, smaller stamps (the "children").
The Rule: The key is that this stamp has a rule: "No matter where you press me, I will always create three new stamps in a specific shape and distance from the original."
The Goal: The computer's job is to look at the first few layers of stamps (the data we have) and figure out exactly what that magical stamp looks like. Once it learns the stamp, it can press it again and again to generate layers of groups that it has never seen before.

3. How They Did It: The "Recursive Rollout"

The authors built a special type of AI (a neural network) to act as this "stamp."

Training: They showed the AI a small tree (say, 3 levels deep).
Learning: The AI tried to guess the rule: "If I have a group here, where should the three new groups go?"
The Test: They told the AI to keep pressing its "stamp" to generate levels 4, 5, 6, and so on, all the way to level 9, without showing it the answers.
The Result: The AI didn't just guess randomly. It kept the pattern consistent. The new groups it created looked geometrically correct and followed the same "family tree" structure as the original data.

4. The Three Tests

To prove this works, they tried it in three different scenarios:

The "Perfect" Test (CFG): They created a fake world where the rules were mathematically perfect. The AI learned the rule instantly and kept generating perfect trees forever. This proved the math works.
The "Fractal" Test (IFS): They used famous fractal shapes (like the Sierpiński triangle). These shapes are generated by repeating rules, but the rules were slightly different from the ones the AI was trained on. The AI still figured out the "spirit" of the rule and drew the fractal correctly, even though it had never seen that specific fractal before.
The "Messy" Test (Images): They used real photos of cats and dogs (from the CIFAR dataset). They grouped the photos into clusters. Real life is messy; the groups aren't perfect fractals. However, the AI still learned a "local rule" that could predict how the groups would split if they were divided further. It didn't just memorize the photos; it learned the geometry of how the photos were related.

5. Why This Matters (According to the Paper)

The paper claims that finite observations can reveal infinite rules.
If you show a computer a small piece of a pattern, it can learn the "local refinement rule" (the stamp) and use it to build a much deeper, more detailed structure than the data it was originally given.

In short: Instead of just sorting a pile of rocks you have on a table, this method teaches the computer the "law of gravity" for those rocks, so it can predict how the pile would look if you had a million more rocks.

What They Don't Claim

They do not claim this will cure diseases or predict the stock market.
They do not claim this works on every type of messy data (if the data is too chaotic or the groups don't follow a pattern, the method might fail).
They focus strictly on the mathematical ability to learn a "recursive refinement rule" and generate deeper hierarchies, not on specific real-world applications like medical diagnosis.

The core takeaway is a shift in perspective: Don't just learn the data; learn the rule that generates the data, so you can imagine the rest of the tree.

Technical Summary: Classification Fields: Arbitrarily Fine Recursive Hierarchical Clustering From Few Examples

1. Problem Statement

Classical hierarchical clustering methods typically operate on a fixed finite sample, returning either a finite partition of the data or a finite dendrogram. This approach is inadequate when the underlying hierarchy of interest is a recursive geometric object with fine-scale refinements that extend beyond the directly observed levels. In such regimes, the number of latent cluster centers ( $k$ ) may grow far beyond the number of observed nodes ( $N$ ), and refinements may continue at depths not present in the training data.

The central question addressed by this work is: Can we learn a local refinement rule from a finite observed prefix of a hierarchy and recursively roll it out to generate substantially deeper, unobserved levels?

Current differentiable or learned approaches to hierarchy typically fit a finite observed tree, embed a fixed structure, or optimize a finite clustering objective. They do not directly learn a refinement mechanism capable of recursive rollout beyond the observed hierarchy.

2. Methodology: Classification Fields

The authors introduce Classification Fields, a metric-geometric model for infinite-depth hierarchical clustering.

2.1 Core Definitions

Classification Field Generator (CFG): A local rule $r: \mathbb{R}^d \to (\mathbb{R}^d)^K$ $r : R^{d} \to (R^{d})^{K}$ that maps a parent center to an ordered, bounded, and separated $K$ $K$ -tuple of child residuals.
- Constraints: For any parent $x$ , the child residuals $\{r_k(x)\}$ must lie in a Euclidean annulus (bounded norm) and be $s_{sep}$ -separated.
- Construction: CFGs can be constructed via orthogonal-diagonal-orthogonal deformations of a fixed reference packing, ensuring the geometric constraints are met.
Classification Field: The infinite-depth hierarchical structure obtained by recursively applying the CFG starting from a root $x_0$ $x_{0}$ and a scale factor $s$ $s$ .
- Structure: The field is a recursively generated metric Directed Acyclic Graph (DAG).
- Components: It generates cluster centers, associated Voronoi cells, and genealogical addresses.
- Metrics: The paper defines three distance metrics on the field:
  1. Tree Metric: Based on combinatorial edge weights (level-dependent).
  2. Point-based Metric: Based on Euclidean distances between centers.
  3. Cell-based Metric: Based on Hausdorff distances between Voronoi cells, scaled by the depth.

2.2 Learning: Classification Field Predictors (CFP)

Given a finite prefix of a hierarchy (observed up to depth $L_{train}$ ), the goal is to learn a Classification Field Predictor (CFP), denoted $\hat{r}$ , which approximates the underlying CFG.

Training Objective: Minimize the squared error between the predicted child residuals and the observed residuals:
$\mathcal{L}(\hat{r} | \mathcal{H}_{train}) = \sum_{l=0}^{L_{train}-1} \sum_{x \in \mathcal{H}_l} \sum_{k=1}^K \| x + s^{l+1}\hat{r}(x)_k - \text{chld}(x)_k \|_2^2$
Recursive Rollout: At test time, $\hat{r}$ is applied recursively from the root. Every generated child becomes a parent for the next level. No ground-truth nodes from unseen levels are used during rollout.

2.3 Theoretical Guarantees

The paper establishes that finite truncations of the learned field converge to the completed infinite-depth field under specific conditions:

Exponential Convergence: Under boundedness, separation, and regularity assumptions, the Hausdorff distance between the depth- $L$ truncation and the completed field decays exponentially with $L$ .
Neural Realizability: The finite refinement windows required to approximate the CFG can be realized by ReLU Multi-Layer Perceptrons (MLPs).
- Complexity: The required width is $O(\varepsilon^{-\gamma})$ and depth is $\tilde{O}(\varepsilon^{-3\gamma/2})$ , where $\gamma = \log K / (-\log s)$ .
- This proves that a finite neural network can act as a finite approximation mechanism for an infinite-depth hierarchical structure.

3. Experimental Validation

The authors evaluate the CFP across three settings of increasing mismatch from the idealized theory:

Matched CFG-Generated Hierarchies:
- Setup: Data generated by a known CFG from the same family used in the theory.
- Result: The CFP maintains the lowest level-wise error across unseen rollout horizons compared to baselines (learnable constant, average residual, and affine predictors). This confirms that the learned local rule remains coherent under repeated self-composition.
Out-of-Family IFS Fractal Hierarchies:
- Setup: Hierarchies generated by Iterated Function Systems (IFS) with smooth nonlinear perturbations (e.g., Sierpiński triangle, Cantor dust, Koch curve). These have exact recursive structure but are not drawn from the parametrized CFG family.
- Result: The CFP achieves lower MSE, Permutation-Invariant Chamfer Distance (PI-CD), and depth distortion than baselines. This demonstrates that the classification field formulation generalizes beyond the matched generator class.
Image-Induced Recursive Clustering Hierarchies:
- Setup: Hierarchies constructed from CLIP embeddings of CIFAR-10 and CIFAR-100 images using a recursive deep clustering method. The refinement rule is approximate, and child slots require canonicalization (ordering) via PCA.
- Result: The CFP significantly outperforms baselines in preserving unordered geometry (PI-CD) and hierarchy-level path metrics (dpt-distortion). This suggests that even in noisy, real-data settings, finite hierarchy prefixes contain learnable refinement mechanisms.

4. Key Contributions

Conceptual Framework: Introduction of Classification Fields as a metric-geometric model for infinite-depth hierarchical clustering, shifting the focus from learning a hierarchy over finite samples to learning a reusable local parent-to-child refinement rule.
Metric Theory: Development of a rigorous metric theory for these objects, including definitions for tree, point, and cell-based distances, and quantitative bounds for the convergence of finite truncations to the completed field.
Neural Realizability: Proof that finite refinement windows can be represented by ReLU predictors, establishing a finite neural rollout mechanism for infinite-depth hierarchical completion with explicit complexity bounds.

5. Significance and Claims

The paper claims that finite hierarchical observations can reveal local refinement rules capable of generating substantially deeper classification fields. The results support the view that hierarchical clustering can be framed as learning a local recursive refinement law rather than constructing a finite combinatorial summary.

The authors position this work as a complementary view to existing hierarchical clustering methods. They explicitly note limitations: the framework assumes a fixed branching factor, sufficient separation, and a reusable refinement process. They acknowledge that extending the framework to noisy, variable-branching, weakly aligned, or non-self-similar hierarchies remains an important direction for future work. The experiments are presented as validation probes rather than claims that every hierarchical clustering problem admits a perfect classification field.