💻 computer science

ImageHD: Energy-Efficient On-Device Continual Learning of Visual Representations via Hyperdimensional Computing

ImageHD is an FPGA accelerator that enables energy-efficient, real-time on-device continual learning of visual data by combining a quantized CNN feature extractor with a hardware-aware hyperdimensional computing architecture that eliminates costly iterative optimization and complex memory hierarchies.

Original authors: Jebacyril Arockiaraj, Dhruv Parikh, Viktor Prasanna

Published 2026-04-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jebacyril Arockiaraj, Dhruv Parikh, Viktor Prasanna

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a smart robot dog that lives on your farm. Every day, it sees new animals: a cow, then a sheep, then a strange new bird. Your goal is for the robot to learn what these animals are right there on the farm, without needing to call a giant supercomputer in the cloud for help.

This is the challenge of On-Device Continual Learning. The robot needs to learn new things instantly, using very little battery power and memory, without forgetting what it learned yesterday.

Most current robot brains are like heavyweight weightlifters. To learn a new animal, they have to stop, go back to the gym (the cloud), lift heavy weights (complex math called "backpropagation"), and come back tired. This takes too long and uses too much battery.

ImageHD is a new, lightweight brain designed specifically for this robot. Here is how it works, explained simply:

1. The Brain: Hyperdimensional Computing (HDC)

Instead of using complex, heavy math, ImageHD uses something called Hyperdimensional Computing.

The Analogy: Imagine trying to remember a friend's face. A normal computer tries to memorize every single freckle and hair strand perfectly (like a high-resolution photo).
ImageHD's Way: It creates a "mental fingerprint" for the animal. It's like a long string of 10,000 light switches (on or off). If the animal is a cow, the switches are in one pattern. If it's a sheep, they are in a different pattern.
Why it's cool: You don't need to memorize every detail. You just check if the new animal's "light switch pattern" looks similar to the ones you already know. It's fast, simple, and very hard to mess up.

2. The Filter: The Quantized CNN

Before the robot can make its "light switch pattern," it needs to see the animal clearly.

The Problem: The camera sees millions of pixels. That's too much data for a small battery.
The Solution: ImageHD uses a Quantized CNN (a type of image filter). Think of this as a sketch artist who draws the animal using only 256 shades of gray instead of millions of colors.
The Magic: Even though the drawing is simpler (lower quality), the robot's brain (HDC) is so good at recognizing patterns that it doesn't even notice the difference. This saves a massive amount of energy.

3. The Organizer: No More "Filing Cabinets"

Old learning systems are like messy libraries. When a new book (data) comes in, they have to build a new shelf, move old books around, and reorganize the whole library (this is called "clustering"). This takes forever.

ImageHD's Trick: It uses a Unified Memory system. Imagine a single, smart whiteboard.
- When a new animal appears, the robot draws its "fingerprint" on the board.
- If it looks like a cow already on the board, it just adds a little note to that cow's spot.
- If it's a totally new animal, it draws a new spot.
- The "Merge" Button: If the whiteboard gets too full, ImageHD has a special "Merge" button. Instead of reorganizing the whole library (which is slow), it simply combines similar drawings into one bigger drawing. This happens instantly and keeps the board from getting cluttered.

4. The Hardware: The FPGA (The Custom Chip)

You can't run this on a standard phone or laptop chip because they are too general-purpose.

The Analogy: A CPU (like in your laptop) is a Swiss Army Knife. It can do everything, but it's not the best at any one thing. A GPU (graphics card) is a Power Drill. It's fast but uses a lot of electricity.
ImageHD's Chip (FPGA): This is like a Custom-Built Factory. The engineers built a factory where every single machine is designed only to do the specific job of "drawing light switch patterns" and "checking similarities."
Because the factory is custom-built, it doesn't waste a single drop of energy or a single second of time. It processes data like a conveyor belt, never stopping.

The Results: Why Does This Matter?

The researchers tested this on a real chip (an FPGA) and compared it to powerful computers (CPUs and GPUs).

Speed: ImageHD was 40 times faster than a standard computer and 5 times faster than a high-end graphics card.
Battery Life: It used 383 times less energy than a standard computer.

In plain English:
If you were building a robot that needs to learn new things while hiking in the woods (where there is no internet and the battery is precious), ImageHD is the brain that lets it learn instantly without getting tired or needing a charger. It turns a heavy, slow learning process into a fast, light, and efficient one.

1. Problem Statement

The paper addresses the critical challenge of On-Device Continual Learning (CL) for visual data. Edge devices (e.g., drones, IoT sensors, robots) require the ability to learn incrementally from non-stationary data streams without relying on cloud retraining or offline batch processing. However, existing CL methods face significant barriers to deployment on resource-constrained hardware:

High Computational Cost: Most methods rely on backpropagation-based updates (e.g., fine-tuning deep networks) or heavy exemplar storage, which incur high latency and energy consumption.
Memory Constraints: Maintaining large buffers for replaying past data or complex memory hierarchies (short-term vs. long-term memory) exceeds the on-chip memory budgets of edge FPGAs.
Hardware Inefficiency: General-purpose GPUs and CPUs are often inefficient for the specific bitwise operations required by emerging CL paradigms, leading to poor energy efficiency and high static power consumption.
Complexity in Existing HDC Systems: Prior Hyperdimensional Computing (HDC) approaches for CL often utilize multi-tier memory structures and spectral graph clustering, which introduce cubic time complexity ( $O(N^3)$ ) and complex control logic unsuitable for streaming hardware.

2. Methodology: ImageHD

The authors propose ImageHD, a domain-specific FPGA accelerator co-designed with a hardware-optimized algorithm to enable efficient, unsupervised continual learning. The system integrates a lightweight neural front-end with an HDC back-end.

A. Algorithmic Innovations

Hybrid CNN-HDC Pipeline:
- Front-end: Uses a pre-trained, INT8-quantized MobileNetV2 to extract visual features. The authors leverage the inherent noise robustness of HDC to tolerate quantization noise, significantly reducing the memory footprint of the feature extractor without sacrificing accuracy.
- Back-end: Features are encoded into high-dimensional binary hypervectors (HVs) using level and position tables.
Unified Exemplar Memory:
- Replaces the complex multi-tier (Short-Term/Long-Term) memory structures of prior work with a single, bounded cluster memory. This simplifies control logic and ensures the system fits within strict on-chip memory limits.
Hardware-Efficient Clustering (kMeans++ vs. Spectral):
- Exploits the quasi-orthogonality of high-dimensional binary vectors. Instead of using computationally expensive spectral clustering ( $O(N^3)$ ), ImageHD employs a Top-M kMeans++ algorithm.
- Process: It initializes centroids via uniform random sampling and Top-M farthest point selection, followed by a small number of refinement iterations using HDC-style bundling (majority voting). This reduces complexity to linear time ( $O(N)$ ).
Streaming Admission & Update:
- Implements a single-pass streaming algorithm where incoming samples are compared against existing clusters. If a sample is novel (low similarity), a new cluster is created; otherwise, the cluster centroid is updated via bundling.

B. Hardware Architecture (FPGA Accelerator)

Implemented on the AMD Zynq ZCU104 FPGA, ImageHD utilizes a streaming dataflow architecture with five specialized compute units:

Pointwise Convolution Unit (PCU) & Depthwise Convolution Unit (DCU): Pipelined units for the inverted residual blocks of MobileNetV2, utilizing tile-based processing and on-chip weight buffering to minimize off-chip memory traffic.
Hyperdimensional Encoding Unit (HEU): A 2D processing-element (PE) mesh that performs parallel binding (XOR) and bundling (accumulation + thresholding) of feature vectors into binary hypervectors. It uses banked on-chip memories for level and position tables.
Hyperdimensional Learning Unit (HLU): Performs streaming similarity search (Hamming distance calculation) between the encoded sample and stored cluster prototypes. It handles online cluster updates and admission decisions.
Cluster Merge Unit (CMU): Executes the hardware-optimized kMeans++ consolidation algorithm. It streams class hypervectors, computes distances, selects centroids via Top-M sampling (using LFSR for randomness), and updates centroids via bundling.
Dataflow: The system operates in a fully pipelined manner, overlapping feature extraction, encoding, similarity search, and cluster management to maximize throughput.

3. Key Contributions

First End-to-End FPGA Accelerator for Unsupervised CL: ImageHD is the first system to integrate a quantized CNN feature extractor with a streaming HDC pipeline specifically for unsupervised continual learning on edge devices.
Algorithm-Hardware Co-Design:
- Replaced $O(N^3)$ spectral clustering with an $O(N)$ kMeans++ strategy tailored for binary hypervectors, achieving an 8.6× algorithmic speedup.
- Introduced a Unified Exemplar Memory to bound memory usage and simplify hardware control.
- Validated the use of INT8 quantization for the CNN front-end, reducing model size by 3.7× while maintaining CL performance.
Streaming Architecture: Designed a specialized dataflow on FPGA that supports massively parallel bitwise operations within tight on-chip resource budgets, eliminating the need for iterative optimization.

4. Experimental Results

The system was evaluated on the CORe50, CIFAR-10, and CIFAR-100 datasets, comparing performance against optimized CPU (AMD EPYC 7763), GPU (NVIDIA RTX A4000), and Edge GPU (Jetson AGX Orin) baselines.

Latency: ImageHD achieved up to 40.4× speedup over CPU and 4.84× speedup over GPU on the CORe50 dataset.
Energy Efficiency: The FPGA accelerator demonstrated up to 383× higher energy efficiency (mJ/image) compared to CPU and 105.1× compared to GPU.
Clustering Performance: Despite using INT8 quantization and kMeans++ instead of spectral clustering, ImageHD maintained competitive accuracy, purity, and NMI scores. The trade-off in accuracy was negligible (<1% reduction in some metrics) compared to the massive gains in speed and efficiency.
Resource Utilization: The design utilized 62% of LUTs and 52% of BRAMs on the ZCU104, demonstrating efficient resource usage for a complex learning pipeline.

5. Significance

ImageHD establishes a practical foundation for real-time, on-device lifelong learning systems. By demonstrating that hyperdimensional computing can be accelerated efficiently on FPGAs without relying on backpropagation or massive memory buffers, the paper:

Enables edge devices to adapt to changing environments (e.g., new object classes in robotics) in real-time.
Solves the "catastrophic forgetting" problem through bounded memory management and efficient cluster merging.
Provides a blueprint for future edge AI hardware that prioritizes energy efficiency and low latency over raw compute power, making continuous learning feasible for battery-powered and latency-sensitive applications.