💻 computer science

VeloxNet: Efficient Spatial Gating for Lightweight Embedded Image Classification

This paper introduces VeloxNet, a lightweight CNN architecture that replaces SqueezeNet's fire modules with spatial gating units to achieve global spatial modeling with fewer parameters, resulting in significant improvements in both classification accuracy and parameter efficiency across three aerial image datasets for embedded disaster monitoring and infrastructure inspection.

Original authors: Md Meftahul Ferdaus, Elias Ioup, Mahdi Abdelguerfi, Anton Netchaev, Steven Sloan, Ken Pathak, Kendall N. Niles

Published 2026-03-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Md Meftahul Ferdaus, Elias Ioup, Mahdi Abdelguerfi, Anton Netchaev, Steven Sloan, Ken Pathak, Kendall N. Niles

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a tiny, battery-powered robot (like a drone) how to look at pictures of disaster zones—like flooded streets or collapsed buildings—and tell you what it sees.

The problem is that these robots have very small brains (limited memory) and run on weak batteries. If you give them a "super-brain" (a massive AI model), they will crash, run out of power, or be too slow to help in an emergency. If you give them a "baby brain" (a tiny model), they might be fast, but they will miss important details and make mistakes.

VeloxNet is a new, clever way to build a robot brain that is both tiny and smart.

Here is the story of how it works, explained simply:

1. The Old Way: The "Local Detective"

For a long time, engineers used a design called SqueezeNet. Think of this like a detective who only looks at a picture through a tiny 3x3 inch window.

How it works: The detective looks at a small patch of the image, makes a guess, moves the window, and looks at the next patch.
The Problem: If the detective sees a puddle of water, they only see the water. They don't realize it's part of a massive flood because they can't see the whole picture at once. To understand the big picture, they have to take many steps and use a lot of memory to remember what they saw in previous steps. This makes the robot slow and heavy.

2. The New Way: The "All-Seeing Eye" (VeloxNet)

The authors of this paper, VeloxNet, decided to replace that tiny window with a magic lens called a Spatial Gating Unit (SGU).

The Analogy: Instead of looking through a tiny window, imagine the detective suddenly gets a pair of glasses that let them see the entire room at once.
How it works: When the robot looks at a picture, this new "lens" allows every part of the image to instantly "talk" to every other part.
- If the robot sees a broken bridge in the top left, the "lens" instantly tells the part of the brain looking at the bottom right, "Hey, that's a bridge! Be careful!"
- It connects the dots across the whole image in a single step, rather than taking many small steps.

3. Why is VeloxNet Better?

The paper tested this new design on three different disaster datasets (AIDER, CDD, and LDD). Here is what happened:

Smaller Size: VeloxNet is 46% smaller than the old SqueezeNet.
- Analogy: It's like swapping a heavy, bulky backpack full of useless tools for a sleek, lightweight fanny pack that holds exactly what you need.
Smarter: Despite being smaller, it got better grades on the tests.
- On one dataset (CDD), it improved accuracy by a huge 30%.
- Analogy: It's like a student who studies less (smaller brain) but gets higher test scores because they understand the connections between facts, rather than just memorizing isolated facts.
Faster: It runs just as fast as the old models, which is crucial for real-time disaster response.

4. The Secret Sauce: "Gating"

How does it stay small while being smart?

The Old Way (Fire Modules): The old design tried to be efficient by "squeezing" information, like trying to stuff a whole elephant into a suitcase. It had to throw away a lot of details to make it fit, which caused mistakes.
VeloxNet (Gated MLP): VeloxNet uses a "gatekeeper." It splits the information in half. One half looks at the data, and the other half decides which parts of the data are important to keep and which to ignore.
- Analogy: Imagine a bouncer at a club. Instead of letting everyone in (which clogs the place) or checking everyone's ID one by one (which is slow), the bouncer has a list of VIPs. He lets the important stuff through instantly and blocks the noise. This saves space and time.

The Bottom Line

VeloxNet proves that you don't need a giant, heavy computer to do smart things. By changing how the computer looks at an image (from a tiny window to a global view) and using a "gatekeeper" to filter information, you can build a robot brain that is:

Light enough to fit on a small drone.
Fast enough to react in an emergency.
Smart enough to spot disasters accurately.

This is a big deal for saving lives, inspecting bridges, and monitoring the environment with small, cheap robots.

1. Problem Statement

Deploying deep learning models on embedded devices (e.g., drones for disaster monitoring, mobile inspection systems) faces strict constraints on model size, memory, and inference latency. While Convolutional Neural Networks (CNNs) are effective for image classification, traditional lightweight architectures like SqueezeNet rely on "fire modules" that use small convolutional kernels (typically $3 \times 3$ ).

Limitation: Fire modules are restricted to local receptive fields, limiting their ability to capture long-range spatial dependencies crucial for tasks like aerial disaster monitoring (e.g., distinguishing a flooded area from a normal road based on global layout rather than local texture).
Trade-off: Existing methods often force a trade-off where reducing model size (parameters/FLOPs) leads to a degradation in accuracy.
Gap: While Gated Multi-Layer Perceptrons (gMLPs) have shown promise in large-scale vision tasks by replacing attention mechanisms, they have not been adapted for parameter-constrained embedded scenarios or specialized domains like aerial imagery.

2. Methodology: VeloxNet Architecture

The authors propose VeloxNet, a lightweight CNN that replaces SqueezeNet's fire modules with gated Multi-Layer Perceptron (gMLP) blocks containing a Spatial Gating Unit (SGU).

Core Components

Architecture Topology: VeloxNet retains the overall macro-architecture of SqueezeNet (initial convolution, three pooling stages, final classifier) but substitutes every fire module with a gMLP block.
Spatial Gating Unit (SGU):
- Instead of convolutions, the SGU models spatial dependencies via learned projections.
- Mechanism: An input feature map $Z$ is split into two channel-wise halves ( $Z_1, Z_2$ ). $Z_2$ is passed through a spatial projection (a linear layer $W_g$ ) and a non-linearity to generate a gating signal. This signal is multiplied element-wise with $Z_1$ (Hadamard product).
- Global Receptive Field: Unlike convolutions limited to $3 \times 3$ neighborhoods, the SGU allows every spatial position to interact with all others in a single layer, providing global spatial modeling.
- Efficiency: The parameter cost scales with the spatial dimension ( $O(n^2)$ ) rather than the channel dimension. Since spatial tokens ( $n$ ) are often smaller than channel dimensions ( $d$ ) in later network stages, this is highly parameter-efficient.
Design Optimizations for Embedded Hardware:
- Fixed Channel Dimensions: Unlike SqueezeNet's variable channel expansion, VeloxNet uses a fixed 156-channel dimension across all blocks to simplify the architecture and avoid squeeze-expansion bottlenecks.
- Simplified Normalization: Uses a single LayerNorm per block instead of multiple.
- Compact Projections: Spatial projection weights are optimized for the specific feature map sizes of the target datasets.

3. Key Contributions

Architectural Integration: Successfully integrated gMLP blocks as drop-in replacements for SqueezeNet fire modules, preserving the network topology while changing the inductive bias from local convolution to global spatial gating.
Efficiency-Oriented Design: Introduced specific design choices (fixed 156 channels, simplified normalization, compact projections) tailored for resource-constrained embedded hardware, moving beyond the large-scale settings of original gMLP research.
Domain-Specific Evaluation: Evaluated the model on three specialized aerial image datasets (AIDER, CDD, LDD) related to disaster response and infrastructure inspection, a domain previously unexplored for gMLPs.
Comparative Analysis: Provided a detailed layer-by-layer comparison demonstrating that gMLP blocks offer superior spatial modeling capacity per parameter compared to fire modules.

4. Experimental Results

The model was evaluated against 11 baselines (including MobileNet variants, ShuffleNet, EfficientNet, and Vision Transformers) on three datasets: AIDER (Emergency Response), CDD (Comprehensive Disaster), and LDD (Levee Defect).

Performance Metrics

Accuracy: VeloxNet achieved the highest Weighted F1 scores on all three datasets:
- AIDER: 81.57% (vs. SqueezeNet's 75.25%; +6.32% gain).
- CDD: 77.46% (vs. SqueezeNet's 46.63%; +30.83% gain).
- LDD: 91.85% (vs. SqueezeNet's 89.34%; +2.51% gain).
Parameter Efficiency:
- 399,366 parameters, which is 46.1% fewer than SqueezeNet (740,970).
- Significantly smaller than MobileNetV1 (87.6% fewer params) and EfficientNet.
Model Size & Speed:
- Model Size: 1.52 MB (smallest among all tested models).
- Inference Speed: ~347–358 FPS, competitive with SqueezeNet and faster than most baselines.
- FLOPs: 461M (higher than ShuffleNet/MobileNetV3 but lower than most other baselines).

Ablation Study

Residual Connections: Removing them caused the largest performance drop (-7.38%), proving essential for training stability.
Spatial Gating (SGU): Removing the SGU reduced F1 by 3.74%, confirming it is the primary driver of accuracy improvement over standard dense layers.
Depth & Channels: The optimal configuration was 8 blocks with 156 channels; reducing depth or channels significantly hurt performance, while increasing channels yielded diminishing returns.

5. Significance and Conclusion

VeloxNet demonstrates that substituting local convolutional modules with spatial gating blocks can break the conventional Pareto frontier of lightweight CNN design.

Key Insight: By enabling global spatial modeling within a single layer without the quadratic cost of self-attention, gMLP blocks capture spatial dependencies (critical for aerial imagery) more effectively than local convolutions, even with fewer parameters.
Impact: The model proves that high accuracy and extreme parameter efficiency are not mutually exclusive for embedded vision tasks. It offers a viable, lightweight solution for real-time disaster monitoring and infrastructure inspection on resource-constrained devices.
Future Work: The authors suggest extending VeloxNet to object detection/segmentation tasks and using Neural Architecture Search (NAS) to further optimize channel dimensions for different hardware constraints.