💻 computer science

FILTR: Extracting Topological Features from Pretrained 3D Models

The paper introduces FILTR, a learnable transformer-based framework that extracts topological information from frozen 3D pretrained encoders to predict persistence diagrams, evaluated using a new synthetic benchmark called DONUT.

Original authors: Louis Martinez, Maks Ovsjanikov

Published 2026-04-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Louis Martinez, Maks Ovsjanikov

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart robot that has spent years studying millions of 3D objects, like chairs, cars, and animals. This robot is incredibly good at recognizing what an object is (a chair vs. a table) or how it looks (smooth vs. rough). But nobody really knows if this robot understands the object's shape structure in a deeper, more mathematical way.

This paper asks a simple question: Does this robot actually "see" the holes, loops, and separate parts of an object, or is it just memorizing surface details?

Here is the breakdown of their findings and new tools, explained with everyday analogies.

1. The Robot's "Blind Spot" (The Problem)

The researchers tested several of these top-tier 3D robots (called "encoders") to see if they could answer two basic questions about a shape:

How many separate pieces is it made of? (Is it one solid ball, or is it a ball with a separate ring floating next to it?)
How many holes does it have? (Is it a solid sphere, a donut with one hole, or a pretzel with three holes?)

The Result: The robots were surprisingly bad at this. Even though they are trained on massive amounts of data, they mostly focus on the "skin" of the object rather than its internal "skeleton" or topology. It's like a person who can describe the color and texture of a donut perfectly but can't tell you that it has a hole in the middle.

2. The New Test Kitchen: DONUT

To test this properly, the researchers couldn't use normal datasets (like a pile of random chairs) because those don't have enough variety in "holes" and "pieces."

So, they built a new training ground called DONUT.

The Analogy: Imagine a 3D printer that can only print specific shapes: donuts, pretzels, and floating islands. The researchers programmed it to print thousands of these shapes, carefully controlling exactly how many holes and separate pieces each one has.
The Goal: This created a "perfect" test where the only variable is the topological structure (the holes and pieces), allowing them to see if the robots could actually learn these concepts.

3. The New Translator: FILTR

Since the robots didn't naturally "speak" the language of holes and loops, the researchers built a new tool called FILTR (Filtration Transformer).

The Analogy: Think of the 3D robot as a person who speaks "Geometry" but doesn't speak "Topology." The researchers built a translator (FILTR) that sits between the robot and the answer.
How it works: The robot looks at the object and gives the translator its "thoughts" (features). The translator then uses a special neural network (like a set of smart filters) to guess the "persistence diagram."
- What is a Persistence Diagram? Imagine a map that lists every hole and loop in an object, noting how "strong" or "long-lasting" that hole is. A tiny, accidental scratch might be a weak hole; a real donut hole is a strong, permanent one. The diagram is a list of these strengths.
The Magic: Even though the robot didn't know the answer on its own, the FILTR translator was able to look at the robot's "thoughts" and successfully predict the map of holes and loops. It's like a translator who can take a person's vague description of a room and draw a perfect architectural blueprint of the doorways and windows, even if the person didn't realize they were describing them.

4. The Key Findings

The Robot is "Implicitly" Smart: The 3D robots do contain some information about holes and loops, but it's hidden deep inside their complex layers. They don't just "know" it; they have to be "unlocked" by the right tool (FILTR).
Speed and Efficiency: Because FILTR uses a robot that is already trained (frozen), it doesn't need to learn everything from scratch. It's like hiring a translator who already knows the language, rather than teaching a new student from zero. This makes the process very fast and efficient.
Generalization: The system worked well even when shown shapes it had never seen before (like moving from a dataset of synthetic donuts to real-world CAD models). This suggests the "translator" learned a universal rule about how to find holes in 3D space.

Summary

The paper is essentially saying: "We found that our best 3D AI models are actually quite 'topologically blind' on their own. However, we built a new, efficient translator (FILTR) that can read their hidden thoughts and accurately map out the holes and separate parts of any 3D object, turning a blind robot into a structural expert."

Technical Summary: FILTR: Extracting Topological Features from Pretrained 3D Models

Problem Statement

Recent advances in self-supervised pretraining for 3D point cloud encoders (e.g., Point-BERT, Point-MAE) have yielded powerful models with strong generalization capabilities on geometric and semantic tasks. However, the structural properties and expressive power of the features learned by these encoders remain poorly understood, particularly regarding topological information. While topological invariants (such as the number of connected components and genus) and multiscale descriptors (persistence diagrams) are critical in fields like material science, protein analysis, and geoscience, existing methods for extracting these features typically rely on classical, decoupled estimation algorithms rather than end-to-end learning.

This paper addresses two primary questions:

Do pretrained 3D point cloud encoders implicitly capture topological information in their learned representations?
Can this information be extracted efficiently to predict persistence diagrams directly from encoder features in a feed-forward manner?

Methodology

The authors propose a three-stage approach to investigate these questions:

1. Benchmarking with DONUT

To evaluate topological understanding, the authors introduce DONUT (Dataset Of maNifold strUcTures), a synthetic benchmark of 29,517 manifold meshes. Unlike existing datasets (e.g., ShapeNet, ModelNet) which are primarily semantic or lack topological diversity, DONUT features controlled variations in:

Number of connected components ( $\beta_0$ ): Ranging from 1 to 6.
Total genus ( $g$ ): Ranging from 0 to 10.
The dataset ensures a balanced distribution of labels and applies geometric transformations to prevent trivial retrieval-based solutions.

2. Probing and Alignment Analysis

The authors evaluate four state-of-the-art pretrained encoders (Point-BERT, Point-MAE, PointGPT, PCP-MAE) using two protocols:

Probing: Trainable linear decoders are attached to the frozen features of each transformer block to predict $\beta_0$ and genus. This tests if topological signals are linearly accessible.
Feature Alignment: Using Centered Kernel Alignment (CKA), the authors measure the similarity between encoder features and vectorized persistence diagrams (using methods like ATOL, Top 128, Landscapes, etc.). This assesses the presence of multiscale topological information without training a predictor.

3. FILTR (Filtration Transformer)

To move beyond analysis to extraction, the authors propose FILTR, a framework that predicts persistence diagrams directly from frozen pretrained encoder features.

Architecture: FILTR adapts the DETR (DEtection TRansformer) architecture. It treats persistence diagrams as unordered sets of points.
Input: Features from a frozen 3D encoder (either the final block or a combination of all blocks) are fed into a transformer decoder via cross-attention.
Output: The decoder predicts a set of persistence pairs $(b_i, d_i)$ and their existence probabilities.
Loss Function: The model is trained with a set-prediction objective comprising:
- Hungarian Matching: To align predicted and ground-truth pairs.
- Reconstruction Loss (MSE): For matched pairs.
- Existence Loss: A binary cross-entropy to distinguish valid pairs from "no-pair" slots.
- Diagonal Regularizer: Forces unmatched predictions toward the diagonal ( $b=d$ ), reducing the need for aggressive thresholding during inference.

Key Contributions

DONUT Benchmark: The introduction of the first dataset explicitly designed with balanced, reliable topological annotations (connected components and genus) for evaluating 3D encoders.
First Topological Analysis of 3D Encoders: The first systematic study quantifying how well modern 3D point-cloud encoders capture topological information. The study reveals that while encoders show limited ability to linearly predict global topology (low probing accuracy), they retain significant multiscale structural information aligned with persistence diagrams (high CKA scores).
FILTR Framework: The first model to predict persistence diagrams directly from pretrained, frozen 3D encoder features in a feed-forward manner. It demonstrates that topological descriptors can be extracted without retraining the backbone or hard-coding algorithmic priors into the network.

Results

Probing Performance: Pretrained encoders perform poorly on direct probing tasks for genus and connected components (e.g., Point-BERT CLS token achieves ~57% accuracy on components, ~26% on genus), suggesting they do not explicitly encode global topology in a linearly separable way.
Alignment: Despite poor probing results, CKA analysis shows consistent alignment between encoder features and vectorized persistence diagrams, particularly in MAE-based models (Point-MAE, PCP-MAE). This indicates that topological signals are present but non-linearly distributed.
FILTR Performance:
- FILTR successfully approximates persistence diagrams from frozen encoders, often matching or surpassing end-to-end trained baselines (like PointNet++ or DGCNN) on out-of-distribution datasets (ModelNet40, ABC).
- The model generalizes to unseen data distributions, though performance varies by dataset (e.g., higher errors on ABC compared to ModelNet40).
- FILTR is computationally efficient, requiring significantly fewer trainable parameters (~~5.7M) compared to end-to-end baselines (~~8M+), as the heavy encoder remains frozen.
- Ablation studies confirm the necessity of the existence loss and diagonal regularizer for stable training and accurate diagram reconstruction.

Significance and Claims

The paper claims that while current 3D pretraining strategies (focused on geometric reconstruction or semantic alignment) do not explicitly optimize for topological invariants, the resulting feature spaces retain sufficient structural information to enable data-driven topological extraction.

Theoretical Insight: The work suggests that 3D encoders implicitly learn local geometric structures that correlate with topology, even if they fail to represent global topological invariants linearly.
Practical Utility: FILTR offers a computationally efficient, feed-forward alternative to classical topological pipelines (which are often slow and decoupled from learning). It enables the extraction of persistence diagrams from raw point clouds using existing foundation models.
Limitations: The authors modestly note that the approach is constrained by the quality and availability of pretrained encoders. In domains lacking strong general-purpose 3D encoders (e.g., graph learning), these conclusions may not transfer. Furthermore, predicting the most persistent (dominant) topological features remains challenging, as these require a global geometric understanding that pretrained models struggle to capture.

The authors conclude that their work provides a foundation for understanding the topological competence of 3D feature extractors and developing fully data-driven approaches for topological descriptor extraction.