🤖 machine learning

Differentiable Lifting for Topological Neural Networks

The paper introduces $\partial$ lift (DiffLift), a general framework that enables end-to-end, differentiable learning of graph liftings to higher-order structures, thereby overcoming the limitations of static, unsupervised methods and significantly boosting the performance of Topological Neural Networks across various benchmarks.

Original authors: Jorge Luiz Franco, Gabriel Duarte, Alexander Nikitin, Moacir Ponti, Diego Mesquita, Amauri H. Souza

Published 2026-08-04

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Jorge Luiz Franco, Gabriel Duarte, Alexander Nikitin, Moacir Ponti, Diego Mesquita, Amauri H. Souza

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer to understand the world by looking at a map of connections. In the world of artificial intelligence, these maps are called graphs, where dots (like people or atoms) are linked by lines (like friendships or chemical bonds). For years, smart computer programs called Graph Neural Networks have been great at reading these maps, but they have a blind spot: they mostly only see who is directly connected to whom. They miss the bigger picture, like how a whole group of friends hangs out together or how a complex shape is formed by many overlapping circles.

To fix this, scientists invented Topological Neural Networks. Think of these as super-powered readers that can look at a graph and say, "Hey, these three dots aren't just friends; they form a triangle!" or "These five dots make a little clique." This ability to see shapes and groups (called higher-order structures) helps the computer understand complex data much better. But here's the tricky part: to see these shapes, the computer first has to be told how to look. It needs a "lifter" to turn the flat map of dots and lines into a 3D structure of shapes. Until now, scientists had to guess which lifting method to use, like trying to guess the right key for a lock without ever seeing the lock. Sometimes the guess works, and sometimes it fails miserably, leaving the computer confused.

This is where a new paper comes in with a clever solution called $\partial$ lift (pronounced "DiffLift"). Instead of guessing the right key, the authors built a smart, learning machine that figures out the perfect way to lift the map for every specific job. Imagine you have a box of Lego bricks (your data). Old methods were like following a rigid instruction manual that said, "Always build a castle," even if you were trying to build a spaceship. $\partial$ lift is like having a robot that looks at your bricks and your goal, then decides, "Okay, for this spaceship, I need to group these bricks into wings, and for that rocket, I need to group them into fins."

The researchers tested this idea on a bunch of different puzzles, from predicting chemical properties to classifying social networks. They found that $\partial$ lift is a game-changer. In many tests, it beat the old, static guessing methods by a huge margin—sometimes improving accuracy by up to 45%. It works by looking at the data, learning which groups of dots are important, and then "lifting" them into the right shapes (like triangles or bubbles) to help the computer solve the problem.

The paper shows that this method works for different types of structures, including hypergraphs (where one line can connect many dots at once) and cell complexes (which are like building blocks of different sizes). The authors suggest that by making the "lifting" process learnable and adaptable, we can stop wasting time on bad guesses and let the AI design its own best way to see the world. While the method does take a bit more computer power to figure out the best shapes, the results suggest it's worth the extra effort, especially for tricky problems where the old methods get stuck. It's a step toward AI that doesn't just follow rules, but understands the shape of the data it's given.

Technical Summary: Differentiable Lifting for Topological Neural Networks

Problem Statement

Topological Neural Networks (TNNs) have emerged as a powerful class of models capable of leveraging high-order structures (e.g., cycles, cliques, simplices) to enhance the expressive power of traditional Graph Neural Networks (GNNs). However, a critical bottleneck exists in the "lifting" process: the transformation of input graphs into topological domains (such as hypergraphs, simplicial complexes, or cell complexes).

Currently, this lifting operation is predominantly unsupervised and static. Methods like cycle lifting, $k$ -hop lifting, or $k$ -nearest-neighbor ( $k$ -NN) lifting rely on predefined heuristics based on graph connectivity or node features. As demonstrated in the paper, the optimal choice of topological domain and lifting procedure is highly data-dependent and task-specific. Static, unsupervised liftings often result in suboptimal topological representations, limiting the downstream performance of TNNs. While differentiable lifting has been explored previously for cell complexes (e.g., DCM), a general framework applicable to various topological domains (including hypergraphs and simplicial complexes) has been lacking.

Methodology: $\partial$ lift (DiffLift)

The authors propose $\partial$ lift, a general, end-to-end differentiable framework for learning graph liftings. Instead of using fixed rules, $\partial$ lift learns to identify and parameterize distributions over candidate higher-order cells for inclusion in the output structure.

Core Mechanism

The framework operates through an iterative, hierarchical sampling process:

Node Embeddings: An arbitrary GNN (e.g., GIN, GPS) computes latent representations ( $z_v$ ) for each node in the input graph.
Candidate Elicitation: Based on these embeddings, the model defines a set of candidate cells.
- For hypergraphs, candidates are formed by selecting $k_v$ nearest neighbors of a node $v$ in the embedding space, where $k_v$ is adaptively sampled from a distribution parameterized by $z_v$ .
- For cell complexes, the process is hierarchical. For dimension $D=1$ , candidate edges are formed similarly to hyperedges. For $D \ge 2$ , candidate cells (e.g., 2-cells) are derived from cycles (or cliques) in the current complex, respecting the hierarchical constraints of the target domain.
Acceptance/Rejection: For each candidate cell $C$ , a neural network (typically an MLP or DeepSet) processes the multiset of node embeddings within $C$ to output an acceptance probability $\phi(z_C)$ .
Stochastic Sampling: A Bernoulli sample determines whether the cell is included in the lifted structure. Gradients are propagated through this discrete sampling step using the straight-through estimator.
Iteration: The process repeats for increasing dimensions ( $D \to D+1$ ) until a maximum dimension $D_{max}$ is reached.

Feature Lifting

Features for the newly formed higher-order cells are computed via a scaled sum projection of the constituent node features, ensuring the lifted structure is compatible with standard TNN message-passing layers.

Key Contributions

General Differentiable Framework: $\partial$ lift is a general framework that enables end-to-end learning of graph liftings across multiple topological domains, including hypergraphs, simplicial complexes, and cell complexes, extending beyond previous work limited to specific domains.
Adaptive Structure Learning: Unlike static methods, $\partial$ lift learns the size and composition of higher-order cells (e.g., adaptive hyperedge sizes) based on task-specific objectives and learned node representations.
Hierarchical Sampling: The method introduces an iterative sampling procedure where lower-dimensional cells inform the generation of higher-dimensional ones, naturally modeling the hierarchical nature of topological objects.
Integration: The framework is designed to be seamlessly integrated into existing TNN pipelines, acting as a learnable preprocessing layer that feeds into off-the-shelf TNN architectures.

Experimental Results

The authors evaluated $\partial$ lift on 12 datasets spanning graph classification (molecular properties) and node classification tasks, using four different TNN architectures (CWN, CXN, CIN, UniGCNII, UniGIN).

Graph Classification: $\partial$ lift consistently outperformed static lifting methods (cycle, $k$ -hop, $k$ -NN, kernel) in 22 out of 24 experiments. It achieved performance gains of up to 45% over static liftings. For instance, on the ZINC dataset, $\partial$ lift reduced the Mean Absolute Error (MAE) from 0.46 to 0.17 when using CWN.
Node Classification: $\partial$ lift achieved competitive or superior performance compared to static liftings and the differentiable baseline DCM. Notably, it significantly outperformed DCM on heterophilic datasets (e.g., Texas). On the Cora dataset, while DCM achieved 80.73%, $\partial$ lift combined with TopoTune achieved 86.82%, though $\partial$ lift with CWN (80.17%) was slightly lower than DCM. Overall, $\partial$ lift demonstrated strong performance across datasets, often outperforming DCM by a large margin.
Robustness: The method demonstrated robustness across different backbone GNNs (GIN vs. GPS), though richer representations (GPS) generally yielded better results.
Efficiency: While $\partial$ lift incurs a modest runtime overhead compared to static methods (due to the sampling and probability computation), it remains computationally feasible. For hypergraphs, the process is embarrassingly parallel; for cell complexes, the cost is dominated by cycle basis computation but remains manageable.

Significance and Claims

The paper claims that $\partial$ lift addresses a fundamental limitation in topological deep learning: the reliance on unsupervised, task-agnostic lifting procedures. By making the lifting process learnable and task-informed, the authors demonstrate that the topological structure of the data can be optimized specifically for the downstream task.

The authors highlight that their approach leads to substantial empirical gains, suggesting that the "optimal" topological domain is not a fixed property of the data but a function of the learning objective. Furthermore, they posit that adaptive liftings can mitigate issues like oversquashing by learning to introduce higher-order cells (e.g., 2-cells) that create shortcuts across bottlenecks in the graph, thereby improving information flow between distant nodes.

The work concludes that while current implementations are limited to 2-dimensional cell complexes due to TNN library constraints, the framework is generalizable to other domains (e.g., point clouds) and represents a significant step toward fully adaptive topological deep learning.