Optimizing Binary and Ternary Neural Network Inference on RRAM Crossbars using CIM-Explorer

This paper introduces CIM-Explorer, a modular toolkit that addresses the limitations of existing RRAM-based CIM tools by providing an end-to-end compiler, simulator, and Design Space Exploration flow specifically optimized for Binary and Ternary Neural Network inference.

The Gist

Imagine you are trying to solve a massive puzzle, but every time you move a piece from your table (memory) to your hands (processor), you have to walk across a crowded room. This is how traditional computers work, and it's slow and energy-hungry. This is called the "von Neumann bottleneck."

To fix this, scientists are building a new kind of computer where the memory and the brain are in the same room. They use a special material called RRAM (Resistive Random Access Memory) that acts like a grid of tiny switches. These switches can be "on" or "off" (or sometimes "half-on"), and they can do math just by letting electricity flow through them. This is called Computing-in-Memory (CIM).

However, these tiny switches are messy. They aren't perfect; they vary from cell to cell, and they can get hot or unstable. Because of this messiness, it's hard to use them for complex math. It's like trying to write a novel with a pen that sometimes skips ink or changes color randomly.

The Problem: Too Many Tools, No Map

Researchers have been building tools to help design these new computers, but they are like having a map for the kitchen, a compass for the bedroom, and a telescope for the garage. You can't use them together easily.

• Some tools only help you write the code (compilers).
• Some tools only help you guess how the hardware will behave (simulators).
• Most tools assume the hardware is perfect or only works with standard 8-bit numbers, which doesn't fit these new "binary" (on/off) switches well.

The Solution: CIM-Explorer

The authors of this paper built CIM-Explorer, a "Swiss Army Knife" for designing these new computers. Think of it as a universal translator and simulator that connects the dots between the software (the AI brain) and the hardware (the messy RRAM switches).

Here is how it works, using simple analogies:

1. The Translator (The Compiler)

Imagine you have a recipe written in French (a standard AI model), but your kitchen only understands English and has very specific, quirky rules.

• CIM-Explorer's compiler takes the French recipe and translates it into English.
• It also rewrites the instructions to fit the quirky kitchen. For example, if the recipe says "add a pinch of salt," and your kitchen only has "salt" or "no salt," the compiler figures out how to combine "salt" and "no salt" to get the right flavor.
• It supports Binary Neural Networks (BNNs) and Ternary Neural Networks (TNNs). Think of these as recipes that only use two or three ingredients (On/Off, or On/Off/Maybe). This makes them perfect for the messy RRAM switches.

2. The Map Makers (The Mappings)

Since the switches are messy, you can't just dump the data on them. You have to choose how to lay it out.

• Analogy: Imagine you are trying to park cars in a bumpy, uneven lot.
  • Option A: Park every car in its own spot (uses a lot of space, but safe).
  • Option B: Park two cars in one spot, one facing forward and one backward (saves space, but if the lot is bumpy, they might crash).
• CIM-Explorer tests all these different parking strategies (called "mappings") to see which one works best for your specific bumpy lot. It tells you: "If your switches are this messy, use Strategy A. If they are less messy, Strategy B saves you space."

3. The Crystal Ball (The Simulator & DSE)

Before you build the actual computer chip (which is expensive), you want to know if it will work.

• CIM-Explorer has a crystal ball (simulator). You can tell it, "What if my switches are 10% inaccurate?" or "What if my power meter (ADC) only reads 3 digits instead of 8?"
• It runs thousands of simulations instantly to show you: "If you use this mapping with these settings, your AI will still be 95% accurate. But if you change this one thing, it drops to 50%."
• This process is called Design Space Exploration (DSE). It's like trying on 100 different outfits in a mirror before buying one, so you don't waste money on the wrong size.

What Did They Find?

Using their new tool, the researchers discovered some cool things:

• The "XNOR" trick: A popular method in the past (using a specific logic gate called XNOR) turned out to be the worst choice for these messy switches.
• Low precision is okay: You don't need perfect, high-definition power meters. Even with very low-resolution readings (like a 3-bit number), the AI can still work surprisingly well if you choose the right mapping.
• Bigger isn't always worse: Usually, bigger AI models are harder to run on messy hardware. But they found that with the right setup, larger models can actually be more robust against the hardware's imperfections than tiny models.

Why Does This Matter?

CIM-Explorer is like a flight simulator for future computers. Instead of building a plane, crashing it, and trying again, engineers can fly it a million times in the computer to find the perfect design.

This toolkit helps researchers:

1. Save time and money by testing designs virtually.
2. Pick the best strategy for their specific hardware.
3. Build smarter, faster, and more energy-efficient AI that can run on tiny chips in your phone, car, or smart home without needing a massive power plant.

In short, they built the ultimate guidebook and testing ground to help us navigate the messy but promising world of next-generation AI hardware.

Technical Summary

1. Problem Statement

Computing-in-Memory (CIM) architectures using Resistive Random Access Memory (RRAM) offer a solution to the von Neumann bottleneck by fusing computation and storage. However, RRAM crossbars suffer from significant non-idealities, including:

• Device Variability: Cycle-to-Cycle (C2C) and Device-to-Device (D2D) variations.
• Physical Constraints: Thermal instability, limited endurance, read disturb effects, and wire resistance.
• Analog Limitations: Input/output noise and non-linear I-V characteristics, which are particularly problematic for Multi-Level Cell (MLC) RRAM.

Due to these challenges, RRAM crossbars are often operated in binary mode (utilizing only High Resistive State [HRS] and Low Resistive State [LRS]), making Binary Neural Networks (BNNs) and Ternary Neural Networks (TNNs) ideal candidates.

The Gap: Existing software tools for RRAM-based CIM are fragmented. They typically focus on only one aspect (compilation, simulation, or Design Space Exploration [DSE]) and often rely on classical 8-bit or 16-bit quantization. There is no unified toolchain that supports the entire design flow for BNNs and TNNs, from early accuracy estimation to code generation for fabricated chips.

2. Methodology: CIM-Explorer

The authors introduce CIM-Explorer, a modular, open-source toolkit designed to optimize BNN and TNN inference on RRAM crossbars. It integrates four key components into a unified DSE flow:

A. TVM-Based Compiler

• Frontend: Built on Larq (an open-source framework for BNN/TNN training) and integrated with Apache TVM. A custom frontend was developed to handle Larq-specific layers (e.g., QuantDense, QuantConv2D) which are not supported in standard TVM.
• Optimization: The compiler partitions Neural Networks (NNs) into Matrix-Vector Multiplications (MVMs) suitable for crossbar execution. It employs scheduling primitives to isolate MVM operations and replace them with function calls to a functional interface.
• Features: Supports arbitrary crossbar sizes, multi-batch processing, and maximizes weight reuse to extend device endurance.

B. Mapping Techniques (Compute Modes)

The toolkit implements various mapping strategies to handle the physical constraints of RRAM (e.g., inability to represent negative conductance or zero resistance):

• Binary Mappings (BNN I–VI): Strategies to map integer weights/activations ($\{-1, +1\}$) to digital/analog crossbar arithmetic.
  • Differential Mode: Uses two cells per weight (one for positive, one for negative parts) to improve robustness against analog errors.
  • Linear-Scaling Mode: Uses a single cell per weight with an offset, reducing cell count but potentially increasing sensitivity to errors.
• Ternary Mappings (TNN I–V): Strategies to map ternary weights/activations ($\{-1, 0, +1\}$) to binary crossbars, either by splitting values into positive/negative parts or using 2-bit binary representations.
• Correction Terms: The system calculates necessary "digital" and "analog" correction terms to compensate for offsets introduced by the mapping (e.g., $G_{min} > 0$).

C. Simulation and Execution Backends

• Functional Interface: Abstracts hardware details, allowing the compiler to generate code that calls a shared library.
• Crossbar Interface: Defines the interaction between the mapper and the hardware/simulator.
• Simulators: Supports various backends, including custom C/C++ simulators for speed and Python callbacks for prototyping.
• ADC Modeling: Includes a simplified Analog-to-Digital Converter model that accounts for clipping (signal exceeding range) and quantization errors based on resolution (bits) and clipping factors.

D. Design Space Exploration (DSE)

CIM-Explorer automates the search for optimal configurations by analyzing the impact of:

• Crossbar parameters (size, variability).
• ADC parameters (resolution, clipping factor).
• Mapping strategies (I–VI for BNN, I–V for TNN).

3. Key Contributions

1. End-to-End Toolkit: The first comprehensive toolchain combining a TVM-based compiler, diverse mapping strategies, and simulators specifically for BNNs and TNNs on RRAM.
2. Unified DSE Flow: Ensures consistency between compilation and simulation by using the same compiler for both DSE and real hardware code generation, eliminating discrepancies caused by differing execution orders.
3. Comprehensive Mapping Analysis: Systematically evaluates six BNN and five TNN mapping techniques, analyzing trade-offs between cell count, cycle count, and robustness to non-idealities.
4. Open Source: The toolkit is publicly available on GitHub, facilitating community adoption and further research.

4. Results and Findings

The authors conducted extensive experiments using models like VGG-7, BinaryNet, and BinaryDenseNet on CIFAR-10 and CIFAR-100.

• Mapping Performance:
  • BNN VI (Differential) consistently achieved the highest accuracy and robustness against cell variability, though it requires 2 cells per weight.
  • BNN V (XNOR), often cited in literature, performed the worst in this study.
  • BNN I & II offered a good balance, outperforming III & IV and eliminating the need for analog correction terms.
• ADC Impact:
  • High accuracy can be maintained even with low ADC resolutions (e.g., 3-bit) for certain mappings (like BNN VI) across a wide range of clipping factors.
  • Differential mappings generally tolerate lower ADC resolutions better than linear-scaling mappings.
• Cell Variability:
  • Variability in the High Resistive State (HRS) had a more significant negative impact on accuracy than Low Resistive State (LRS) variability, largely due to the asymmetry of Gaussian distributions when negative currents are impossible.
  • TNNs showed higher baseline accuracy than BNNs but were less robust against HRS variability because the "0" state (mapped to HRS) occurs more frequently.
• Scalability:
  • Larger models (e.g., BinaryDenseNet37) were found to be less sensitive to non-linearities than smaller models with similar baseline accuracy.
  • For large models, an ADC resolution of just 3 bits was sufficient to maintain accuracy within 1% of the baseline.

5. Significance

• Design Guidance: CIM-Explorer provides critical insights for hardware architects, demonstrating that larger models may be preferable for RRAM CIM due to their robustness against non-idealities, and that 3-bit ADCs may be sufficient for high-performance inference, significantly reducing power consumption.
• Methodological Advancement: By integrating compilation and simulation, the tool prevents the "simulation-reality gap" common in CIM research, ensuring that DSE results are directly applicable to fabricated chips.
• Community Resource: The open-source nature of the toolkit lowers the barrier to entry for researchers exploring BNN/TNN inference on emerging memory technologies, accelerating the development of energy-efficient AI hardware.