In-Memory ADC-Based Nonlinear Activation Quantization for Efficient In-Memory Computing

This paper proposes Boundary Suppressed K-Means Quantization (BS-KMQ), a novel nonlinear quantization method that suppresses boundary outliers to optimize analog-to-digital converter resolution in in-memory computing, achieving significant improvements in quantization accuracy, area efficiency, and energy performance across various deep learning models.

The Gist

Here is an explanation of the paper using simple language and everyday analogies.

The Big Problem: The "Traffic Jam" in Computer Brains

Imagine a modern computer as a busy city. The processor (the brain) is a genius chef who needs to cook meals (process data), and the memory (the pantry) is where all the ingredients are stored.

In a traditional computer (called a "Von Neumann" architecture), the chef has to constantly run back and forth to the pantry to grab ingredients. This running back and forth is slow, wastes a lot of energy, and creates a traffic jam. This is known as the "Memory Wall."

In-Memory Computing (IMC) is like building the pantry inside the kitchen. The chef can cook right where the ingredients are. This is super fast and energy-efficient.

However, there's a catch: To cook perfectly, the chef needs to measure ingredients very precisely. In the digital world, this means converting analog signals (continuous waves of electricity) into digital numbers (0s and 1s). This conversion is done by a device called an ADC (Analog-to-Digital Converter).

If the ADC is too simple (low resolution), the chef guesses the measurements, and the meal tastes bad (the AI makes mistakes). If the ADC is too complex (high resolution), it takes up too much space and uses too much battery, defeating the purpose of saving energy.

The Specific Issue: The "Crowded Edge" Problem

Deep learning networks (like the brains behind self-driving cars or chatbots) have a weird habit. When they process data, they often pile up a massive amount of information right at the edges of their range (near zero or near the maximum limit).

Think of a classroom where 90% of the students are sitting in the very front row and the very back row, leaving the middle empty.

• Old Method (Linear Quantization): Imagine the teacher tries to divide the room into equal-sized zones. They put a line right down the middle. But because everyone is crowded at the edges, the "middle" zones are empty, and the "edge" zones are so packed that the teacher can't tell who is who. The result? A lot of confusion and bad grades (high error).
• The Paper's Solution: The authors realized that trying to measure the empty middle is a waste of time. Instead, we should ignore the extreme outliers (the students sitting on the floor or on the ceiling) and focus our measuring tools on the students actually sitting in the seats.

The Solution: BS-KMQ (The "Smart Sorter")

The paper introduces a new method called Boundary Suppressed K-Means Quantization (BS-KMQ).

1. Suppression (The Bouncer): Before sorting the data, the system acts like a bouncer. It kicks out the extreme outliers (the data points that are too high or too low due to hardware limits or the nature of the math).
2. Smart Clustering (The Party Planner): Instead of dividing the room into equal squares, the system looks at where the people actually are. It creates "zones" that are smaller where the crowd is dense and larger where the crowd is sparse.
3. The Result: This creates a much more accurate map of the data with fewer measurement points.

Analogy: Imagine you are taking a photo of a crowd.

• Linear Method: You take a photo with a standard lens. The people in the front are blurry, and the people in the back are tiny.
• BS-KMQ: You use a zoom lens that focuses perfectly on the main group of people and ignores the few people standing on the roof or the street. The resulting photo is crystal clear, even though you used less "film" (fewer bits).

The Hardware: The "Reconfigurable Ruler"

To make this work in real life, the authors built a special hardware chip.

• The Old Way: Previous chips used a "ruler" with fixed markings. If the data changed, the ruler was still the same, leading to bad measurements.
• The New Way (IM NL-ADC): The authors built a reconfigurable ruler inside the memory itself.
  • It can change the spacing of its markings on the fly.
  • It is incredibly small. The authors say the "ruler" takes up only 3.3% of the space of the whole kitchen, whereas previous designs took up nearly 27%.
  • It's like having a ruler that can shrink or stretch its inches depending on what you are measuring, all while fitting in your pocket.

The Results: Faster, Cheaper, Smarter

When they tested this new system on famous AI models (like ResNet-18 and DistilBERT):

1. Accuracy: The AI made far fewer mistakes. In some cases, the accuracy improved by 66% compared to the old linear method.
2. Efficiency: The system became 24 times more energy-efficient. It's like getting a car that gets 24 times better gas mileage without changing the engine.
3. Speed: It ran 4 times faster.

Summary

This paper solves a major bottleneck in AI hardware. By realizing that AI data is "clumped" at the edges and ignoring those clumps, the authors created a smarter way to measure data. They built a tiny, flexible, in-memory ruler that allows computers to run complex AI models with much less power and space, without sacrificing accuracy.

In one sentence: They taught the computer to stop measuring the empty space and start measuring the crowded space, resulting in a faster, cheaper, and smarter AI brain.

Technical Summary

Here is a detailed technical summary of the paper "In-Memory ADC-Based Nonlinear Activation Quantization for Efficient In-Memory Computing."

1. Problem Statement

In-Memory Computing (IMC) architectures aim to overcome the "memory wall" of von Neumann systems by performing computations directly within memory arrays. However, maintaining high model accuracy in IMC typically requires high-resolution Analog-to-Digital Converters (ADCs), which dominate system energy, area, and latency.

• The Conflict: To reduce hardware costs, IMC accelerators often use low-resolution (3–6 bit) ADCs with linear (uniform) quantization.
• The Issue: Neural network activations are highly non-uniform. Operations like ReLU and hardware-imposed clamping cause activations to accumulate near distribution boundaries (e.g., near zero or at saturation limits).
• Consequence: Existing nonlinear (NL) quantization methods (Lloyd–Max, CDF-based, standard K-means) struggle with these boundary outliers. They either produce irregular step sizes that are hardware-unfriendly, are overly sensitive to outliers, or suffer from unstable clustering near distribution tails. This leads to suboptimal decision levels and severe accuracy loss at low bit widths.

2. Methodology

The authors propose Boundary Suppressed K-Means Quantization (BS-KMQ) coupled with a Reconfigurable In-Memory Nonlinear ADC (IM NL-ADC).

A. Algorithm: BS-KMQ

BS-KMQ is a hardware-aware quantization scheme designed to generate optimal quantization levels for low-bit IMC. It operates in two stages:

1. Robust Statistical Calibration:
   • The network processes calibration batches.
   • To handle outliers caused by ReLU and clamping, the algorithm discards the top and bottom 0.5% of activation values (retaining the central 99%).
   • It computes batch-wise minimum ($b_{min}$) and maximum ($b_{max}$) and updates global boundaries ($g_{min}, g_{max}$) using an Exponential Moving Average (EMA).
2. Boundary-Suppressed Clustering:
   • All samples are clamped to the global range $[g_{min}, g_{max}]$.
   • Samples saturating exactly at the boundaries ($g_{min}$ or $g_{max}$) are removed from the clustering pool.
   • Standard K-means clustering is applied only to the remaining "interior" samples to generate $2^b - 2$ centers.
   • The global boundaries are added back as the extreme quantization centers.
   • Reference Generation: The learned centers are converted into reference levels ($R_i$) for the ADC. The ADC performs a "floor" operation (comparing input against $R_i$) rather than explicit nearest-center rounding, which is more hardware-efficient.

B. Hardware: Reconfigurable IM NL-ADC

The paper introduces a novel SRAM-based macro architecture to implement BS-KMQ efficiently:

• Dual 9T SRAM Bitcell: The core uses a decoupled 9T bitcell supporting ternary multiplication (inputs $\times$ weights). It features separate read paths for positive and negative inputs.
• In-Memory Reference Generation: Unlike traditional designs that require separate arrays to generate ramp voltages (causing high area overhead), this design uses the same dual 9T bitcells for both MAC operations and ADC reference generation.
  • Initial Ramp: Generated by activating negative read word lines (RWL-).
  • Nonlinear Steps: Generated by activating positive read word lines (RWL+) with varying numbers of bitcells to create custom step sizes ($R_{i+1} - R_i$).
• Reconfigurability: The architecture supports dynamic precision scaling (1–7 bits) for inputs, weights, and outputs without complex analog circuitry.
• Calibration: A zero-crossing calibration technique adjusts the initial ramp voltage to mitigate process variations and ensure the ADC crosses the zero point accurately.

3. Key Contributions

1. BS-KMQ Algorithm: A novel quantization method that explicitly suppresses boundary outliers before clustering. It achieves 3×–8× lower Mean Squared Error (MSE) compared to linear, Lloyd–Max, CDF, and standard K-means methods under 3-bit precision.
2. Area-Efficient IM NL-ADC: A reconfigurable (1–7 bit) in-memory ADC architecture.
   • It achieves a 7× area improvement over prior NL-ADC designs (reducing NL-ADC area overhead from ~27% to 3.3% of the MAC array).
   • It eliminates the need for separate ramp generation arrays, significantly reducing silicon footprint.
3. Robustness: SPICE simulations in a 65nm process confirm the design is robust against process variations (SS, TT, FF corners), with error standard deviation increasing by only 1.2× under worst-case conditions due to replica biasing.
4. System-Level Performance: Demonstrates significant speed and energy gains over existing IMC accelerators.

4. Results

The method was evaluated on CNNs (ResNet-18, VGG-16, Inception-V3) and a Transformer (DistilBERT).

• Quantization Error: BS-KMQ consistently yields the lowest MSE. On DistilBERT, it achieved up to 35× lower MSE than linear quantization.
• Accuracy (Post-Training Quantization - PTQ): Compared to linear quantization, BS-KMQ improved accuracy by:
  • 66.8% (ResNet-18)
  • 25.4% (VGG-16)
  • 66.6% (Inception-V3)
  • 67.7% (DistilBERT)
• Accuracy with Fine-Tuning: After low-bit fine-tuning, the method maintained competitive accuracy with minimal loss (0.3%–1.2%) using only 3/3/4/4-bit NL-ADC levels.
• System Efficiency (ResNet-18 on CIFAR-10):
  • Throughput: 2.0 TOPS.
  • Energy Efficiency: 31.5 TOPS/W.
  • Improvements: Compared to state-of-the-art IMC accelerators, this design offers up to 4× speedup and 24× energy efficiency improvement.

5. Significance

This work bridges the gap between the statistical requirements of deep learning (non-uniform activation distributions) and the physical constraints of IMC hardware (low-resolution ADCs).

• Hardware-Software Co-Design: By tailoring the quantization algorithm (BS-KMQ) specifically for the capabilities of the proposed in-memory ADC, the authors avoid the accuracy penalties typically associated with low-bit quantization.
• Scalability: The elimination of peripheral ramp-generation circuits and the use of standard SRAM bitcells make the solution highly scalable and compatible with existing CMOS processes.
• Impact: The results suggest that BS-KMQ is a viable path toward deploying highly accurate, energy-efficient neural network inference on edge devices using low-cost, low-resolution IMC accelerators.