🤖 AI

A Multi Method Importance and Performance Efficiency Analysis of Topological Metrics for Natural Visibility Graph Based Cyber Attack Detection

This study demonstrates that applying a consensus-based importance analysis to Natural Visibility Graph metrics allows for the selection of a compact subset of three topological features that significantly improves cyber-attack detection accuracy and reduces computational runtime by over 96% compared to using the full set of 21 metrics.

Original authors: Ali Melih Kanca, Ilker Turker

Published 2026-10-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ali Melih Kanca, Ilker Turker

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the digital world, network traffic flows like a constant river of data, carrying everything from harmless web browsing to sophisticated cyber-attacks. To protect systems, security experts use computer programs to watch this flow, looking for patterns that signal danger. For years, these programs have relied on counting simple things, like how many packets of data pass through a door or how long a connection lasts. However, researchers have begun to realize that the shape of the data itself might hold more clues than the numbers alone. Imagine taking a line of numbers representing a data stream and turning it into a map where each point is a dot and lines connect dots that can "see" each other without being blocked by anything in between. This creates a unique web, or graph, that preserves the rhythm and structure of the original data. By studying the shape of these webs—how tightly the dots cluster together, how many paths exist between them, and how central certain points are—scientists can uncover hidden signatures of malicious activity that standard counting might miss.

A team of researchers at Karabuk University in Turkey set out to test just how useful these shape-based clues are for catching cyber-attacks. They started with a massive collection of network records containing both normal traffic and various types of attacks. Instead of feeding the raw data directly into a computer program, they first transformed small chunks of the data into these visual webs. From each web, they extracted twenty-one different measurements, or topological metrics, to describe its structure. Some of these measurements looked at how connected the dots were, others examined the average distance between points, and some analyzed how the dots grouped into communities. The researchers then used a powerful type of artificial intelligence, a convolutional neural network, to learn how to distinguish between the webs created by normal traffic and those created by attacks.

The central question the team wanted to answer was whether all twenty-one of these shape measurements were actually necessary. In many scientific fields, researchers often assume that gathering more data leads to better results, but in this case, extracting every single measurement takes a significant amount of time and computing power. The team suspected that some of these measurements might be redundant or less helpful than others. To find out, they applied four different methods to rank the importance of each measurement. One method looked at how much the computer's confidence dropped when a specific measurement was scrambled; another compared the real measurements against random noise to see which ones stood out; a third method systematically removed the least useful ones to see what remained; and a fourth method used a mathematical approach to explain exactly how much each measurement contributed to the final decision. By combining the results of these four distinct approaches, they created a single, agreed-upon ranking of the twenty-one metrics.

The results of this analysis revealed a clear hierarchy. The measurements that described how tightly the points in the web clustered together proved to be the most valuable. Specifically, the median, standard deviation, and mean of these clustering values were ranked as the top three most important indicators. In contrast, other measurements, such as those describing the longest paths or the overall size of the web, were ranked much lower. The researchers then tested this finding by training their computer program using only the top-ranked measurements, gradually reducing the number of inputs from all twenty-one down to just the top three. They compared the performance of these smaller sets against the full set of twenty-one.

The outcome was surprising. When the computer program was trained using only the top three clustering measurements, it performed better than when it was trained with the full set of twenty-one. The simplified model correctly identified attacks with an accuracy of 97.148 percent, compared to 95.999 percent for the full model. It also achieved a higher score in balancing the detection of different attack types. More importantly, this improvement came with a massive gain in speed. The full set of measurements took nearly fifteen thousand seconds to process through the entire experiment, while the top three measurements required only about five hundred and eighty-nine seconds. This represents a reduction in computing time of over ninety-six percent. The study suggests that by focusing on the most informative structural clues and discarding the rest, security systems can become both faster and more accurate.

The researchers noted that this result does not mean that all other measurements are useless, but rather that for this specific type of data and this specific computer model, the extra information was not helping and may have even been distracting. The top three measurements, which describe the local clustering of the data points, seemed to contain the essential signature of the attacks. The study concludes that a compact, carefully selected set of features can outperform a larger, unfiltered collection. While the findings are specific to the dataset and methods used in this experiment, they offer a clear path forward for building more efficient cyber-defense systems. By understanding which structural details matter most, future tools can be designed to detect threats with greater speed and precision, without the burden of processing unnecessary data.

Technical Summary: Multi-Method Importance and Performance–Efficiency Analysis of NVG-Based Cyber-Attack Detection

Problem Statement

Accurate identification of malicious network activities is critical for modern cybersecurity. While Machine Learning (ML) and Deep Learning (DL) have demonstrated efficacy in Intrusion Detection Systems (IDS), their performance relies heavily on how network traffic is represented. Graph-based representations, specifically Natural Visibility Graphs (NVGs), offer an alternative to conventional flow-level analysis by transforming sequential traffic observations into graph structures that preserve ordering and structural variation.

However, a significant challenge remains: NVG-based analysis generates a large set of topological descriptors (e.g., centrality, path length, clustering coefficients). Not all of these metrics contribute equally to cyber-attack classification, and extracting a comprehensive set increases computational costs. There is a lack of systematic investigation into which specific NVG-derived topological metrics are most informative for classification and whether a reduced, compact subset of these metrics can maintain or improve predictive performance while significantly lowering computational overhead.

Methodology

The study employs a multi-method approach to evaluate 21 NVG-derived topological metrics using the CSE-CIC-IDS2018 dataset.

1. Data Preprocessing and Graph Construction:

Dataset: A balanced subset of 100,000 network-flow records covering 15 classes (benign and various attacks) was used, constructed via class-wise sampling without synthetic oversampling.
Sliding Window: Data was segmented into overlapping frames (100 observations, 50% overlap) with attack-prioritized majority voting for labeling.
NVG Construction: Each numerical traffic feature within a frame was independently transformed into a Natural Visibility Graph. Two observations are connected if the straight line between them is not intersected by intermediate observations.
Metric Extraction: 21 topological metrics were extracted from each feature-specific NVG, grouped into:
- Centrality (closeness, betweenness, degree, eigenvector)
- Path length (mean, std, median, min, max)
- Clustering coefficient (mean, std, median, min, max)
- Community/Global structure (modularity, diameter)
- Degree statistics (mean, std, median, min, max)

2. Importance Analysis:
To determine metric relevance, four complementary methods were employed:

SHAP (SHapley Additive exPlanations): Used GradientExplainer on the trained CNN to quantify feature contributions.
Grouped Permutation Importance: Measured performance decrease (weighted F1-score) when metrics were permuted.
Boruta: Compared original features against randomized shadow features using a Random Forest classifier.
Recursive Feature Elimination (RFE): Iteratively removed less informative features using a Random Forest classifier.

3. Consensus Ranking:
The rankings from the four methods were integrated using a Consensus Ranking strategy. The Consensus Score ( $CS_j$ ) for each metric was calculated as the arithmetic mean of its ranks across the four methods. Lower scores indicated higher importance.

4. Performance–Efficiency Evaluation:
Based on the consensus ranking, six configurations were evaluated using a multi-input Convolutional Neural Network (CNN) with stratified 5-fold cross-validation:

Full21: All 21 metrics.
Reduced Sets: Top15, Top10, Top7, Top5, and Top3 (retaining the highest-ranked metrics).
Metrics: Classification performance (Accuracy, Weighted F1-score, MCC) and computational cost (total runtime) were compared.

Key Contributions

Multi-Method Importance Analysis: A systematic evaluation of 21 NVG-derived topological metrics using four distinct importance analysis techniques (SHAP, Permutation, Boruta, RFE) to address the variability in feature relevance assessment.
Consensus-Based Evaluation: The integration of disparate rankings into a unified Consensus Ranking to identify the most robustly important metrics.
Performance–Efficiency Trade-off Analysis: An empirical investigation demonstrating that importance-guided metric reduction can yield compact NVG representations that maintain or exceed the predictive performance of full metric sets while drastically reducing computational costs.

Experimental Results

Consensus Ranking:
The analysis revealed that clustering-coefficient-based descriptors were the most informative. The top three metrics in the Consensus Ranking were:

avg_clustering_coeff_median (Consensus Score: 1.25)
avg_clustering_coeff_std (Consensus Score: 3.00)
avg_clustering_coeff_mean (Consensus Score: 4.25)

Other high-ranking metrics included eigenvector_mean, degree_centrality_mean, and modularity. Notably, individual methods produced varying rankings (e.g., eigenvector_mean ranked 1st by SHAP but 14th by RFE), validating the necessity of a multi-method consensus approach. Boruta selected all 21 metrics, confirming their general relevance but failing to differentiate relative importance.

Classification Performance:

Full21 Baseline: Achieved 95.999% accuracy, 95.521% weighted F1-score, and an MCC of 0.9549.
Top3 Configuration: Retaining only the top three clustering metrics achieved the highest observed mean performance:
- Accuracy: 97.148%
- Weighted F1-score: 97.055%
- MCC: 0.9675
- This configuration outperformed the Full21 baseline across all metrics.

Computational Efficiency:
The reduction in metrics led to substantial runtime savings. The Top3 configuration reduced the total runtime from 14,961.39 seconds (Full21) to 589.22 seconds, representing a 96.06% reduction in computational cost.

Significance and Claims

The paper claims that under the evaluated experimental setting, the relevance of NVG-derived descriptors is not uniformly distributed, and relying on a single importance method can lead to inconsistent rankings. By employing a consensus strategy, the study identifies that local connectivity characteristics (specifically clustering coefficients) are particularly informative for cyber-attack classification.

The primary significance lies in the finding that a compact, importance-guided representation (Top3) can achieve higher observed mean predictive performance than the full set of 21 metrics while reducing computational requirements by over 96%. This suggests that retaining a larger number of metrics does not necessarily yield superior results and that removing less informative or redundant descriptors may have a regularizing effect, allowing the model to focus on more discriminative structural information.

The authors maintain a modest stance, noting that these findings are specific to the CSE-CIC-IDS2018 dataset, the CNN architecture used, and the experimental protocol. They explicitly state that the Top-K results reflect a comparative analysis within the current framework and call for future work to assess the robustness of these metric subsets across different datasets and models, as well as to perform metric-level runtime profiling.