🔢 mathematics

Hybrid Topological Data Analysis and LSTM Networks for Enhanced Network Intrusion Detection Using CIC-IDS2017 Dataset

This paper proposes a hybrid network intrusion detection system that integrates Topological Data Analysis with Long Short-Term Memory networks to achieve near-perfect detection accuracy (AUC and F1-score of 1.000) on the CIC-IDS2017 dataset by effectively capturing both topological and temporal characteristics of network traffic.

Original authors: Amar Jeet, Bhaskar Ranjan Karn, Dinesh Kumar

Published 2026-07-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Amar Jeet, Bhaskar Ranjan Karn, Dinesh Kumar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a security guard trying to spot a thief in a massive, bustling train station. Most security systems work like a Wanted Poster: they have a photo of a known criminal (a known virus or attack pattern). If the person walking by matches the photo, they get stopped. But what if the thief is wearing a disguise, or it's a brand-new type of criminal the police have never seen before? The "Wanted Poster" system fails.

This paper proposes a new kind of security guard that uses two different superpowers to catch these tricky thieves: Shape-Shifting (Topology) and Time-Travel (LSTM).

Here is how their new system works, broken down into simple concepts:

1. The Two Superpowers

Superpower A: The "Shape" Detective (Topological Data Analysis)
Imagine looking at a crowd of people not as individuals, but as a cloud of dots.

Normal Traffic: The dots might form a smooth, solid cloud.
A DDoS Attack (Overwhelming the system): The dots might clump together into a dense, tight ball.
A Port Scan (Looking for open doors): The dots might spread out in a sparse, systematic grid, like someone checking every door on a street.

The authors use a mathematical tool called Topological Data Analysis (TDA) to look at the "shape" of these clouds. They don't care about the specific details of every single person; they care about the loops, holes, and connections in the crowd.

The Analogy: Think of a coffee mug and a donut. To a normal eye, they look different. But to a topologist, they are the same shape because they both have exactly one hole. Similarly, this system looks for the "holes" and "loops" in network traffic. Even if a hacker tries to disguise their attack (change the "color" of the dots), the fundamental "shape" of the attack often stays the same. This makes it very hard for hackers to hide.

Superpower B: The "Time" Detective (LSTM Networks)
Network traffic isn't just a snapshot; it's a movie. A hacker doesn't just knock on one door; they knock on 100 doors in a specific order over a few seconds.

LSTM (Long Short-Term Memory) is a type of AI that is really good at watching movies. It remembers what happened a moment ago to understand what is happening right now.
The Analogy: If you see someone walking normally, that's fine. But if you see someone walking normally, then suddenly sprinting, then stopping, then sprinting again in a weird pattern, the LSTM remembers that sequence and says, "That doesn't look like a normal commute."

2. The Hybrid Team-Up

The paper suggests that using just the "Shape" detective or just the "Time" detective is good, but using both together is perfect.

They built a system where:

The Shape Detective looks at a chunk of traffic and says, "This looks like a dense ball of connections (a DDoS attack)!"
The Time Detective watches the sequence and says, "And they did it in a rapid, repeating pattern!"
They combine their notes into a single report.

3. The Test Drive (The Experiment)

The team tested this new system on a famous dataset called CIC-IDS2017. Think of this dataset as a massive, realistic simulation of a train station that includes:

Over 2.8 million "passengers" (network connections).
14 different types of "thieves" (attacks like DDoS, brute force, botnets, etc.).
A mix of normal people and criminals.

The Results:
The results were incredibly high. The paper claims their hybrid system achieved:

100% Accuracy in distinguishing between normal traffic and attacks in their tests.
It beat other standard methods (like Random Forests or Isolation Forests) which had slightly lower scores.
Even when they tested it on specific, rare types of attacks (like "Botnets" or "Infiltration"), it still caught them.

4. Why This Matters (According to the Paper)

It's Hard to Trick: Because the system looks at the "shape" of the data, hackers can't easily hide by just changing small details. The overall structure of their attack gives them away.
It Understands Time: It doesn't just look at one moment; it understands the story of the attack as it unfolds.
It's a New Way of Thinking: Instead of just memorizing known bad patterns, this system learns to recognize the geometry and rhythm of an attack.

5. The Catch (Limitations Mentioned)

The authors are honest about the downsides:

It's Heavy: Calculating these "shapes" takes a lot of computer power. It's like trying to analyze the shape of a cloud while it's raining; it's computationally expensive.
It Needs Tuning: You have to set the rules carefully (like how big the "time windows" are) for it to work best.
It's New: They only tested it on one specific dataset (CIC-IDS2017). They admit they need to test it on other modern datasets to be sure it works everywhere.

Summary

In short, this paper says: "If you want to catch network hackers, don't just look at their faces (known patterns). Look at the shape of their crowd and the rhythm of their footsteps. By combining a math tool that sees shapes with an AI that remembers time, we can spot intruders with near-perfect accuracy."

Technical Summary: Hybrid Topological Data Analysis and LSTM Networks for Enhanced Network Intrusion Detection

1. Problem Statement

Network Intrusion Detection Systems (NIDS) face significant challenges in modern cybersecurity environments, including high-dimensional feature spaces, temporal dependencies in network traffic, class imbalance between normal and malicious operations, and the need for real-time processing. Traditional signature-based systems struggle with zero-day exploits and dynamic threat landscapes. While deep learning, particularly Long Short-Term Memory (LSTM) networks, has shown promise in modeling sequential data, static classifiers often fail to capture the underlying geometric structures of network traffic. Furthermore, adversaries frequently employ obfuscation and evasion strategies that can bypass standard statistical approaches. This research addresses the need for a system that can simultaneously capture both the structural (topological) and temporal characteristics of network traffic to improve anomaly detection.

2. Methodology

The paper proposes a novel hybrid architecture that integrates Topological Data Analysis (TDA) with LSTM networks via a Multi-Layer Perceptron (MLP) fusion network. The methodology is structured as follows:

A. Data Preprocessing

The study utilizes the CIC-IDS2017 dataset, containing over 2.8 million labeled bidirectional network flows with 77 features (after preprocessing) and 14 attack categories (including DDoS, Brute Force, Web Attacks, Infiltration, and Botnets). The preprocessing pipeline includes:

Data cleaning (removing duplicates, infinite, and NaN values).
Min-Max normalization of numerical features.
Stratified subsampling to maintain class ratios while managing computational costs for TDA.
Construction of overlapping time windows (window size $w=50$ , step size $s=25$ ) to capture temporal sequences.

B. Topological Data Analysis (TDA) Component

The TDA module extracts structural features from time windows of network data:

Point Cloud Construction: Each time window is treated as a point cloud in high-dimensional space.
Persistent Homology: A Vietoris-Rips filtration is constructed using Euclidean distance. This captures topological features (connected components and loops) that persist across multiple scales.
Betti Curves: The persistence diagrams are converted into functional representations known as Betti curves ( $\beta_0$ and $\beta_1$ ), which describe connectedness and loop patterns. These curves are discretized into fixed-length feature vectors (200 points).
Feature Extraction: The resulting vectors ( $z_{TDA}$ ) represent topological invariants that are robust to noise and continuous deformation.

C. LSTM Component

The LSTM module processes the sequential network data to capture temporal dependencies:

Architecture: Two stacked LSTM layers (64 and 32 hidden units) with dropout (0.2) and batch normalization.
Function: The gating mechanism of LSTMs captures long-range temporal dependencies, solving the vanishing gradient problem common in standard RNNs.
Output: The module produces a temporal feature representation ( $z_{LSTM}$ ).

D. Hybrid Fusion

The core innovation lies in the Feature Fusion stage:

The topological features ( $z_{TDA}$ ) and temporal features ( $z_{LSTM}$ ) are concatenated.
A learned MLP fusion network processes this combined representation.
The final output layer uses a Softmax activation for binary classification (Normal vs. Attack).

3. Key Contributions

The paper outlines four primary contributions:

Novel Architecture: A hybrid TDA+LSTM framework that integrates topological feature extraction with deep temporal modeling for network intrusion detection.
Mathematical Formulation: A rigorous definition of Betti curve extraction and its fusion with deep learning models, specifically utilizing persistent homology to generate noise-resilient features.
Comprehensive Evaluation: A thorough experimental assessment on the CIC-IDS2017 benchmark, including ablation studies, 5-fold cross-validation, and statistical significance testing.
Feature Analysis: An in-depth examination of how topological features (specifically Betti curves) complement temporal representations across various attack categories.

4. Experimental Results

The proposed model was evaluated against baselines including Isolation Forest, TDA+Random Forest, Traditional SVM, and standalone LSTM.

Performance Metrics: The hybrid TDA+LSTM model achieved an AUC of 1.000 and an F1-Score of 1.000 on the CIC-IDS2017 dataset.
Cross-Validation: 5-fold cross-validation yielded a mean AUC of $1.000 \pm 0.000$ and a mean F1-score of $0.999 \pm 0.001$ .
Comparative Analysis:
- TDA+LSTM Hybrid: F1 = 1.000
- TDA + Random Forest: F1 = 0.994
- Isolation Forest: F1 = 0.835
- Standalone LSTM: F1 = 1.000 (Note: The paper notes the hybrid model achieves competitive performance with the standalone LSTM but emphasizes the added value of topological interpretability).
Ablation Study:
- TDA-only (MLP): F1 = 0.990
- LSTM-only: F1 = 1.000
- Hybrid: F1 = 1.000
- The study demonstrates that while LSTM alone performs exceptionally well, the topological features contribute complementary information, particularly in handling class imbalances and providing geometric insights.
Computational Cost: The hybrid model requires more training time (1.4 minutes) and memory (469 MB) compared to traditional SVM (<0.1 min, 89 MB) or TDA+RF (0.3 min, 145 MB), indicating a trade-off between computational overhead and feature richness.

5. Significance and Claims

The paper claims that the hybrid approach offers a new paradigm for network intrusion detection by blending the learning power of deep neural networks with the mathematical rigor of topological data analysis.

Synergistic Effects: The combination of TDA and LSTM provides robust performance, with topological features offering stability against noise and perturbations that often hinder statistical methods.
Interpretability: Unlike "black box" deep learning models, the TDA component provides geometric insights into attack patterns (e.g., dense connectivity in DoS attacks vs. sparse exploration in PortScans).
Robustness: The model demonstrates effectiveness across diverse attack categories, including Brute Force, Botnets, and Web Attacks, with perfect classification in the tested windows.
Limitations: The authors modestly acknowledge limitations, including increased computational overhead for large-scale installations, sensitivity to parameter settings (window size, filtration radius), and the need for further validation on other modern datasets (e.g., CICIoT2023, UNSW-NB15) before generalizing claims.

In conclusion, the research validates that integrating topological invariants with temporal sequence modeling enhances the detection of complex network intrusions, offering a method that is both highly accurate and structurally interpretable.