🤖 AI

TopoAlign: Topology-Aware Visual Representation Alignment

This paper introduces TopoAlign, a topology-aware framework that leverages mapper graphs and force-directed optimization to visually compare and analyze the structural alignment of neural network representations across models and layers, offering insights beyond traditional geometric methods.

Original authors: Xinyuan Yan, Rita Sevastjanova, Mennatallah El-Assady, Bei Wang

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Xinyuan Yan, Rita Sevastjanova, Mennatallah El-Assady, Bei Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have two different chefs (neural networks) trying to cook the same meal (process the same data, like words or images). Even if they end up with a delicious dish, they might organize their ingredients and cooking steps very differently.

TopoAlign is a new tool that helps us see how these two chefs organize their ingredients, not just by looking at the final taste, but by mapping the entire kitchen layout.

Here is a simple breakdown of how it works, using everyday analogies:

1. The Problem: Two Different Maps

When computers learn, they turn data (like the word "apple" or a picture of a cat) into long lists of numbers. These lists are like high-dimensional coordinates.

The Old Way: Previous tools tried to compare these lists by measuring the distance between individual numbers. It's like trying to compare two cities by measuring the distance between every single house. It gives you a number, but it doesn't show you the shape of the city or how the neighborhoods connect.
The TopoAlign Way: This tool uses something called a Mapper Graph. Imagine taking a blurry, high-resolution photo of a city and turning it into a simple subway map.
- Nodes (Stations): These are clusters of similar items (e.g., a station for "fruits," another for "vehicles").
- Edges (Tracks): These show where the clusters overlap (e.g., a track connecting "fruits" and "red things" because apples are both).

2. The Solution: Aligning the Subway Maps

TopoAlign takes the "subway map" of Chef A and the "subway map" of Chef B and tries to lay them side-by-side so you can see where they match and where they differ.

It does this in three main steps:

Step 1: Pulling the Maps Together (Global Alignment)

Imagine you have two subway maps drawn on separate pieces of paper. They are both messy and oriented differently. TopoAlign uses a "magnetic force" to gently pull the two maps together.

If a station on Map A represents "dogs" and a station on Map B also represents "dogs," the tool pulls them close together.
This creates a coordinated view where you can instantly see if the two chefs organized their "dog" stations in the same part of the map or in completely different corners.

Step 2: Finding Matching Neighborhoods (Local Alignment)

Once the maps are pulled together, the tool looks for specific neighborhoods that match. It uses a technique called Bubble Sets.

Think of this as drawing a glowing, fuzzy bubble around a group of stations on both maps.
The color of the bubble tells you how much the ingredients inside overlap (Jaccard similarity).
The smoothness of the bubble's edge tells you how well-organized that neighborhood is.
This helps experts quickly spot: "Oh, Chef A has one big messy station for 'animals,' but Chef B has split that into three clean stations: 'cats,' 'dogs,' and 'birds.'"

Step 3: Zooming In with a "Membrane" View

Sometimes, you need to see exactly which ingredients are shared between two matching neighborhoods.

TopoAlign uses a Membrane View. Imagine two parallel glass walls. On the left wall is Chef A's neighborhood; on the right is Chef B's.
Strings (edges) connect the specific items that are shared between the two walls.
If the strings are tangled, it means the chefs are confused about how to group things. If the strings are straight and clear, the chefs agree perfectly.
You can also "merge" nodes to simplify the view, like zooming out on a map to see the big picture, or zooming in to see the details.

3. What Did They Find? (The Case Studies)

The authors tested this on two types of "chefs":

Language Models (BERT): They watched how a model changed as it learned over time (from 2 days of training to 6 days).
- The Insight: Early on, the model was messy, grouping words by how they looked (e.g., "for" and "four" together). Later, it learned to group them by meaning. TopoAlign showed exactly when and how the model stopped being confused and started organizing words by their actual definitions.
Multimodal Models (CLIP): They compared how a model understands images vs. text.
- The Insight: The model might see a picture of a "woman with a fire hydrant" as one thing, but the text description might group "fire hydrants" and "women" separately. TopoAlign visualized these "crossing" paths, showing exactly where the image understanding and text understanding disagreed.

Summary

TopoAlign is like a translator that turns complex, invisible computer math into two side-by-side subway maps. It helps experts see:

Where two models agree (they have the same stations).
Where they disagree (one model split a station, the other merged them).
How the organization changes as the model learns (the map gets cleaner and more logical over time).

It doesn't just tell you if two models are similar; it shows you the shape of their thinking.

Technical Summary: TopoAlign

Problem Statement

Neural networks encode inputs as high-dimensional vectors (representations) that capture task-relevant structures and semantics. A fundamental task in analyzing these models is representation alignment, which measures the degree to which different models, layers, or training conditions produce similar representations for the same inputs. This analysis is critical for understanding model learning dynamics, informing architecture design, guiding model selection, and assessing robustness.

Existing approaches to quantify alignment primarily rely on geometric properties, such as pairwise similarity matrices, neighborhood-based comparisons (e.g., k-nearest neighbors), or cluster similarity. While these methods provide aggregated scalar scores, they often obscure nuanced structural patterns and offer limited insight into the global organization of representation spaces. Furthermore, visual analytics tools that do exist often focus on pointwise distances or require manual inspection of correspondences, lacking systematic methods for comparing the topological structures of representations across models.

Methodology: TopoAlign

To address these limitations, the authors propose TopoAlign, a topology-aware visual analytics framework that compares model representations by analyzing Mapper graphs, a technique from Topological Data Analysis (TDA). The framework operates on a multi-scale, top-down workflow:

1. Input and Graph Construction

Given a shared input dataset $D$ and two models (or layers) $f_1$ and $f_2$ , the system extracts representations $Z_1$ and $Z_2$ . It constructs two Mapper graphs, $G_1$ and $G_2$ , using identical parameter configurations (filter function, interval overlap, and clustering algorithm like DBSCAN) to ensure comparable resolution.

Nodes represent clusters of data points.
Edges denote non-empty intersections (overlaps) between these clusters.
The system uses the $\ell_2$ -norm as the default filter function to capture magnitude variations in representations.

2. Joint Force-Directed Optimization (Global Alignment)

To enable direct visual comparison, TopoAlign constructs a joint Mapper graph ( $G_{joint}$ ) by introducing inter-graph edges between nodes in $G_1$ and $G_2$ that share common data points.

Objective: Compute coordinated 2D layouts for both graphs simultaneously.
Mechanism: The system minimizes an energy function $L(p)$ $L (p)$ that balances:
- Intra-graph forces: Standard attraction (spring-like) and repulsion (particle-like) forces to preserve the internal structure of each individual graph.
- Inter-graph alignment forces: A term that pulls nodes sharing data points closer together. The strength of this force is weighted by the Jaccard similarity of the data subsets and controlled by a parameter $\lambda$ .
Result: A side-by-side visualization where structurally similar regions are spatially aligned, revealing global topological differences (e.g., branching, loops, component separation).

3. Automatic Local Alignment Discovery

The system automatically identifies local correspondences between the two graphs using alignment-aware spectral clustering on the joint graph.

Clustering: Nodes are partitioned into groups where intra-graph connections are dense and inter-graph connections link corresponding regions.
Visualization: Each identified local alignment pair is visualized using Bubble Sets.
- Shading Opacity: Encodes Jaccard similarity (content alignment).
- Boundary Waviness: Encodes the Silhouette score (structural coherence).
This allows users to quickly identify and assess the quality of local alignments without manual inspection.

4. Alignment Motif Querying

Local alignment pairs are categorized into five canonical motifs based on how components map between the two subgraphs:

One-to-one: Consistent feature representation.
Fan-out: A single component splits into multiple (indicating refinement or disentanglement).
Fan-in: Multiple components merge into one (indicating aggregation).
Crossing: Complex, partially overlapping reorganization.
Vanishing/Appearance: Features present in only one model.
Users can filter and explore alignments based on these patterns.

5. Multi-Scale Fine-Grained Inspection (Membrane View)

To inspect detailed correspondences without visual clutter, TopoAlign introduces a membrane-inspired visualization.

Aggregation: Components are iteratively merged into "supernodes" guided by conditional entropy of inter-edges. This strategy avoids trivial collapse and reveals coherent alignment structures at intermediate levels of abstraction.
Visualization: The two subgraphs are displayed as parallel layers connected by inter-edges. Supernodes are rendered as ovals containing their internal structure, allowing users to explore alignment at multiple scales (from coarse supernodes to fine-grained data points).
Integration: The system integrates LLM-based agents (e.g., GPT-4o) to summarize textual content of aligned items, aiding semantic interpretation.

Key Contributions

Globally Aligned Mapper Visualizations: A joint force-directed optimization framework that produces coordinated embeddings for direct comparison of global topological structures.
Automatic Cross-Graph Region Correspondence: An alignment-aware clustering approach that identifies structurally matching subgraphs automatically.
Interpretable Local Alignment Characterization: A Bubble Sets-based visualization coupled with motif-based querying to reveal local patterns (splitting, merging, etc.) and assess alignment quality.
Multi-Scale Alignment Inspection: A membrane-inspired visualization technique that supports detailed exploration of fine-grained structural correspondences while minimizing visual clutter.
Integrated Interactive System: A unified workspace enabling exploratory analysis from global structures to local patterns.

Results and Case Studies

The authors demonstrate TopoAlign through case studies on language models (BERT-Base, BERT-Tiny) and multimodal models (CLIP):

Training Dynamics (BERT): Comparing BERT-Base at epoch 2 vs. epoch 6 revealed that fine-tuning progressively disentangles semantic categories. Early epochs showed mixed components, while later epochs exhibited clear separation. Specific motifs like "Fan-out" (splitting mixed categories) and "Fan-in" (merging similar categories) were identified, correlating with improved prediction accuracy.
Layer Evolution (BERT): Comparing Layer 3 vs. Layer 12 showed a transition from lexical organization (grouping by word form) in early layers to semantic organization (grouping by category) in later layers.
Model Capacity (BERT-Base vs. BERT-Tiny): The smaller model exhibited more entangled label structures. TopoAlign identified specific semantic features (e.g., the word "like") that the smaller model failed to disentangle compared to the larger model, suggesting targeted fine-tuning strategies.
Multimodal Alignment (CLIP): Comparing image and caption representations revealed modality-specific differences in semantic granularity. For instance, "Fan-out" patterns indicated that captions captured finer semantic distinctions than images, while "Crossing" patterns highlighted how different modalities group concepts (e.g., "woman" vs. "fire hydrant") differently.

Expert Feedback

A preliminary study with five deep learning experts (PhD students and postdocs) yielded positive feedback:

Experts found the global alignment layout effective for identifying correspondences without manual node inspection.
Automatic alignment discovery and motif categorization were deemed intuitive and helpful for understanding high-level patterns.
The Bubble Sets visualization was praised for selecting structures based on boundary fuzziness.
Suggestions for improvement included adding semantic annotations directly to Bubble Sets and refining the interpretation of supernode connections in the membrane view.

Significance and Claims

The paper claims to be the first work to systematically compare neural network representations using topological abstractions (Mapper graphs).

Novelty: Unlike prior work that relies on geometric distances or manual comparison, TopoAlign provides a systematic, automated framework for multi-scale topological comparison.
Insight Generation: The framework reveals meaningful structural phenomena—such as feature splitting, merging, and disentanglement—that are often obscured by aggregated metrics.
Utility: It supports model interpretation, robustness analysis, and the understanding of learning dynamics across training stages, layers, and modalities.
Modesty: The authors acknowledge that their automatic alignment detection is designed to surface potentially meaningful correspondences rather than a single optimal solution, serving as an entry point for exploratory analysis. They also note that internal representation alignment does not always correlate directly with model performance, though performance metrics can help contextualize observed patterns.