💻 computer science

Energy-Efficient CNN Acceleration with MSDF Digit-Serial Arithmetic on FPGA

This paper proposes a merged multiply-add (MMA) architecture using MSDF digit-serial arithmetic on FPGA to overcome latency bottlenecks in U-Net convolutional layers, achieving up to 15.14 GOPS/W energy efficiency and a 9× reduction in power consumption compared to existing implementations.

Original authors: Muhammad Usman, Yousef Sadegheih, Dorit Merhof

Published 2026-06-25

📖 4 min read☕ Coffee break read

CC0 1.0

Original authors: Muhammad Usman, Yousef Sadegheih, Dorit Merhof

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a massive puzzle, but instead of looking at the whole picture at once, you have to build it one tiny piece at a time, from the most important piece down to the least important. This is essentially what a computer does when it processes medical images to find tumors, using a system called a U-Net.

This paper introduces a new, super-efficient way to build the "engine" that does this puzzle-solving work, specifically designed for a type of computer chip called an FPGA (which is like a Lego set for electronics that you can reprogram).

Here is the breakdown of their invention using simple analogies:

The Problem: The "Waiting Room" Bottleneck

Traditionally, when these chips do math (multiplying and adding numbers), they use a method called MSDF (Most-Significant-Digit-First). Think of this like a high-speed train that only stops at the most important stations first.

The Good News: It's very compact and saves space.
The Bad News: The train has a long "waiting room" at the start. Before it can drop off the first passenger (the first digit of the answer), it has to sit there for a few cycles.
The Compounding Issue: In a complex calculation, you often have to multiply numbers and then add them up. In old designs, you had a "Waiting Room" for the multiplication, and then another "Waiting Room" for the addition. If you chain these together, the delays pile up, making the whole process slower.

The Solution: The "Merged" Assembly Line

The authors built a new machine called the MMA (Merged Multiply-Add) unit.

The Analogy: Imagine a factory. In the old way, you had a "Multiplication Station" where workers waited in line, finished their task, and then walked over to a separate "Addition Station" where they waited in another line.
The Innovation: The authors smashed these two stations together into one giant, streamlined assembly line. Now, the multiplication and addition happen in a single, continuous flow.
The Result: Instead of waiting in two separate lines, the data only waits in one short line. This removes the "startup delay" that used to slow everything down.

How It Works in Practice

The U-Net architecture (used for things like spotting brain tumors in MRI scans) needs to look at a 3x3 grid of pixels at a time.

The Old Way: You might process these pixels one by one or in small, clunky groups.
The New Way: The authors built 16 parallel assembly lines (called Kernel Processing Blocks). Imagine 16 teams of workers all solving different parts of the puzzle at the exact same time. Because their "Merged" machines are so efficient, they can process 16 output pixels simultaneously without getting bogged down by delays.

The Results: Speed vs. Energy

The paper compares their new chip against standard computers (CPUs), powerful graphics cards (GPUs), and other FPGA designs.

The CPU: Like a very smart, general-purpose librarian. It's accurate but slow and uses a lot of electricity to do the heavy lifting.
The GPU: Like a massive army of workers. It's incredibly fast but eats up a huge amount of power (like a stadium full of lights).
The New FPGA Design: Like a team of highly specialized, energy-efficient robots.
- Speed: It's not quite as fast as the massive GPU, but it's significantly faster than the CPU and other FPGA designs.
- Energy: This is the big win. The new design is 7 times more energy-efficient than the CPU and 2.7 times better than the GPU.
- The Bottom Line: It delivers about 15 units of work for every unit of energy, compared to the CPU's 1.9 units.

Why This Matters (According to the Paper)

The authors emphasize that this is perfect for edge applications—devices that need to run on batteries or in places where power is limited, like portable medical diagnostic tools. By merging the math operations and running them in parallel, they created a system that is fast enough to be useful but efficient enough to run without draining a battery or overheating a small device.

In short: They took a math process that used to have a lot of "waiting time" between steps, glued those steps together into one smooth motion, and ran 16 of them at once. The result is a medical imaging accelerator that is much greener and more efficient than what we currently use.

Technical Summary: Energy-Efficient CNN Acceleration with MSDF Digit-Serial Arithmetic on FPGA

Problem Statement

Accurate brain tumor segmentation using the U-Net architecture is critical for medical imaging but computationally intensive. Implementing U-Net on standard CPUs and GPUs consumes significant energy and resources, posing challenges for embedded and edge deployment where power efficiency is paramount. While Field-Programmable Gate Arrays (FPGAs) offer a promising platform for acceleration, existing digit-serial arithmetic techniques, specifically Most-Significant-Digit-First (MSDF), face a specific bottleneck: initial latency.

In traditional MSDF implementations, inner product calculations are performed using a cascaded structure of a multiplier followed by an adder tree. Each stage (multiplier and each level of the adder tree) introduces a fixed startup delay ( $\delta$ ). In cascaded operations, these delays accumulate, significantly increasing the total latency before the first output digit is produced. This overhead undermines the throughput potential of digit-serial processing, particularly in convolutional layers where inner products are dominant.

Methodology

The authors propose a novel hardware architecture that fuses multiplication and addition into a single Merged Multiply-Add (MMA) unit to eliminate the cumulative latency of cascaded components.

Core Architecture

Merged Multiply-Add (MMA) Unit:
- Instead of separate multiplier and adder stages, the MMA unit performs bitwise multiplication and accumulation simultaneously within a unified pipeline.
- Operation: The unit processes 32 input channels ( $T_N = 32$ ) in parallel. It accepts 8-bit quantized activations (streamed serially, MSDF) and parallel 8-bit weights.
- Logic: At each clock cycle, the $b$ -th bit of the 32 activation channels is ANDed with the corresponding 8-bit weights. The resulting partial products are immediately accumulated using a carry-propagate adder tree.
- Residual Handling: To support digit-serial computation, a "residual" (the lower bits of the partial sum) is carried over to the next cycle. The adder tree accommodates $T_N + 1$ inputs (32 partial products + 1 residual), requiring $\lceil \log_2(33) \rceil$ stages.
- Redundant Number System: The design employs a Signed Digit (SD) redundant number system with radix-2 and digits $\{1, 0, -1\}$ . This allows the algorithm to correct results in subsequent cycles if an early output digit is tentative, providing the necessary flexibility for MSDF arithmetic without requiring full precision at every step. The authors utilize Inverted Encoding of Negabit (IEN) to simplify sign handling.
System-Level Organization:
- Kernel Processing Block (KPB): A $3 \times 3$ convolution kernel is implemented using a group of 9 MMA units (one for each spatial location). The outputs of these 9 units are summed via an MSDF adder tree to produce a single partial sum for one output pixel.
- Parallelism: The system instantiates 16 parallel KPBs, allowing the computation of 16 output pixels simultaneously per clock group.
- Target Application: The design is optimized for the U-Net architecture, specifically targeting convolutional layers with $3 \times 3$ kernels and channel counts that are multiples of 32.

Key Contributions

The paper outlines four primary contributions:

Novel MMA Architecture: A fused multiply-add unit that combines digit-serial multiplication and accumulation into a single pipeline, reducing per-iteration latency by eliminating the separate delays of multipliers and adder trees.
Parallel Spatial Processing: An architecture where MMA units process multiple spatial input depths in parallel, significantly increasing the throughput of convolution operations.
Energy-Efficient FPGA Implementation: A complete accelerator for U-Net convolutional layers implemented on an FPGA, demonstrating applicability to resource-constrained edge applications.
Comprehensive Evaluation: A comparative analysis against bit-parallel, bit-serial, conventional MSDF, GPU, and CPU implementations across latency, throughput, area, and energy efficiency.

Results

The proposed design was implemented on a Xilinx Zynq-7020 SoC using Verilog RTL and evaluated against a U-Net model quantized for fixed-point inference. The accelerator operates at 100 MHz.

Throughput: The design achieves 52.95 GOPS. This represents a $1.07\times$ improvement over conventional bit-parallel designs, a $4.36\times$ improvement over bit-serial designs, and a $2.52\times$ improvement over contemporary MSDF-based accelerators. While the GPU achieved higher absolute throughput (385.99 GOPS), it did so at a significantly higher power cost.
Energy Efficiency: The accelerator delivers 15.14 GOPS/W. This is:
- $2.7\times$ better than the GPU (5.51 GOPS/W).
- $7.8\times$ better than the CPU (1.93 GOPS/W).
- $5.0\times$ better than the MSDF-based FPGA implementation (3.01 GOPS/W).
Energy Consumption: The energy consumed per operation batch is 186.20 mJ, which is $3.8\times$ lower than the GPU and $10.8\times$ lower than the CPU.
Area Efficiency: The design achieves 17.43 GOPS/Slice (scaled by $10^{-4}$ ), which is $1.65\times$ higher than bit-parallel designs and nearly $6.7\times$ higher than MSDF-based accelerators.

Significance and Claims

The paper claims that the Merged Multiply-Add (MMA) approach effectively addresses the initial latency bottleneck inherent in cascaded MSDF arithmetic. By fusing operations, the design achieves a streamlined latency per iteration that is shorter than the combined latency of conventional units.

The authors position this work as a solution for resource-constrained, latency-sensitive edge applications, particularly in medical imaging (brain tumor segmentation) and computer vision. The results suggest that while the accelerator operates at a lower frequency than a CPU, its architectural efficiency yields an order-of-magnitude improvement in energy efficiency. The authors conclude that this approach is highly suitable for embedded systems where energy constraints are critical, offering a viable alternative to power-hungry GPU/CPU inference.

Future work, as modestly stated by the authors, involves adding early termination mechanisms to skip unnecessary operations and scaling the architecture with more tiles to support more complex CNNs.