Block-Sparse Pruning Compresses Models Without Reducing Inference Latency in Intrusion Detection Networks

Imagine the internet as a massive, bustling city where millions of digital messages zip between buildings every second. To keep this city safe, security guards (called Intrusion Detection Systems) stand at the gates, scanning every package for signs of trouble. For a long time, these guards relied on a giant, heavy book of "known bad guys" to spot threats. But hackers are clever; they invent new tricks every day that aren't in the book yet. So, scientists started teaching the guards to think for themselves using "Deep Learning"—a type of computer brain that learns patterns by looking at thousands of examples.

However, there's a catch. These super-smart computer brains are huge and hungry. They need massive amounts of memory and energy, which is fine for a giant server in a data center, but terrible for the tiny, battery-powered devices that actually guard our homes and routers (like the little computers inside your smart thermostat or a network switch). Scientists have been trying to shrink these brains down without breaking them, using a technique called "pruning." Think of pruning like trimming a hedge: you cut away the branches (or in this case, the unused connections inside the computer brain) to make it smaller and faster. The big assumption in the tech world has always been: "If you cut away 75% of the connections, the brain will be 75% lighter and run much faster."

But what if that assumption is wrong? What if, after all that cutting, the brain still runs just as slowly because the way it's built doesn't actually let it skip the empty spots? That is the mystery this paper sets out to solve.

---

The Great Pruning Experiment

In this study, a researcher named Ishan Mishra decided to put this "cutting makes it faster" idea to the ultimate test. He didn't just run the numbers on a powerful supercomputer; he took his computer brain and put it on a Raspberry Pi 5—a tiny, affordable computer board that looks like a green circuit board and is the kind of device you might find in a real-world security system.

He built a "Multi-Classifier Deep Neural Network," which is a fancy way of saying a computer brain with three different layers of thinking, each checking for trouble. He trained this brain on a dataset called NSL-KDD, which is like a practice test full of old, known cyber-attacks. To make sure the brain didn't get confused by having too many examples of one type of attack and too few of another, he used a trick called SMOTE to balance the training data, essentially creating fake examples of rare attacks so the brain would learn them properly.

Then came the pruning. He took the trained brain and applied block-sparse pruning, which is a specific way of cutting out 74.55% of the connections (weights) inside the network, leaving them as zeros. The goal was to see if this "empty" brain would run faster and take up less space than the original, full brain.

The Surprise: Cutting Didn't Speed Things Up

Here is the twist: The pruning didn't make the brain run any faster.

The researcher measured how long it took for the computer to make a single decision (inference latency).

• The original, full brain took 0.0103 ± 0.0008 ms (milliseconds).
• The pruned, "empty" brain took 0.0101 ± 0.0002 ms.

Statistically, these two numbers are twins. They are effectively the same. The paper used a special statistical test called TOST (Two One-Sided Tests) to prove that the difference was so tiny it didn't matter in the real world. The result? Pruning did not reduce the time it took to make a decision.

Why? The paper explains that while the brain had fewer connections, the software running on the Raspberry Pi (called TFLite) wasn't smart enough to skip the empty spots. It was like having a chef who is told to ignore 75% of the ingredients on the counter, but the chef still has to walk over to every single spot, look at it, and decide "oh, that's empty," before moving on. The walking time (latency) didn't change because the chef still had to check every spot.

The File Size Surprise

The researcher also checked how big the final file was when he saved the brain to run on the device. He expected the pruned version to be much smaller. Instead, both the full brain and the pruned brain ended up being exactly 105,992 bytes (about 103.51 KB).

This happened because the software used to prepare the file for the device (a process called quantization) squished the numbers down to their smallest size anyway, making the "empty" spots in the pruned version irrelevant. So, in this specific setup, pruning didn't save any storage space either.

The Real Problem: Unstable Brains

If pruning didn't make it faster or smaller, did it hurt the brain's ability to spot hackers? Mostly, no. The overall accuracy stayed very close:

• The full brain got 82.3% of the known attacks right.
• The pruned brain got 81.2% right.

However, the paper found something very interesting about how the pruned brain failed. When the researchers ran the experiment five times with different random starting points (called "seeds"), they noticed that the pruned brain was unstable.

For some types of attacks, like the smurf attack, the full brain was consistently good (getting it right about 89% of the time). But the pruned brain was a rollercoaster. Depending on which random seed was used, it might be great (85% right) or terrible (11% right). It was like a student who sometimes aces a test and sometimes fails it completely, depending on how they woke up that morning.

The paper also found that both the full and pruned brains struggled with certain attacks like back and guess_passwd, getting them almost 0% right. This wasn't because of pruning; it was because the training data was too unbalanced, causing the brain to just guess "normal" for everything it wasn't sure about. This failure happened every single time, no matter the seed.

What About New, Unknown Attacks?

The test also included 3,750 records (16.6% of the test set) containing attack types the brain had never seen before. Since the brain wasn't trained on these, it couldn't name them. Instead, the researchers checked if the brain at least flagged them as "suspicious" instead of ignoring them.

• The full brain flagged about 36.6% of these unknown attacks.
• The pruned brain flagged about 37.0%.

Again, there was no real difference. Pruning didn't help the brain spot new, unknown tricks, nor did it make it worse at them.

The Takeaway

This paper is a reality check for anyone trying to shrink AI models for small devices. It proves that just cutting out connections (pruning) doesn't automatically make things faster if the software running the model isn't designed to take advantage of those empty spots.

If you want a faster model on a tiny device, you can't just rely on pruning alone. You either need to use special software that knows how to skip the empty spots, or you need to use a different kind of pruning that actually removes whole chunks of the brain (structured pruning) rather than just making holes in it.

The study concludes that while pruning can make a model smaller in theory, in the real world of today's standard software, it doesn't buy you speed or space. And if you do use it, you have to be careful: it might make your model's performance on specific threats much more unpredictable, swinging between great and terrible depending on how it was trained.