💻 computer science

Multibit neural inference in a N-ary crossbar architecture

This paper presents a simulation framework for N-ary crossbar architectures using multi-state magnetic tunnel junctions to demonstrate energy-efficient multibit neural inference, achieving 94.48% MNIST accuracy while identifying weight quantization as the primary error source and determining an optimal number of cell states to minimize total matrix-vector multiplication error.

Original authors: Anatole Moureaux, Anthony Lopes Temporao, Flavio Abreu Araujo

Published 2026-05-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Anatole Moureaux, Anthony Lopes Temporao, Flavio Abreu Araujo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a traditional computer as a busy librarian who has to run back and forth between the stacks (memory) and the desk (processor) to find books and read them. Every time the librarian moves, they waste energy and time. This is the "von Neumann bottleneck" that slows down modern AI.

This paper proposes a new way to do things: In-Memory Computing (IMC). Instead of running back and forth, imagine the librarian is the bookshelf. You ask a question, and the books themselves rearrange their pages to give you the answer instantly. This is much faster and uses less energy.

Here is a simple breakdown of what the researchers did and found, using everyday analogies:

1. The "Magic Grid" (The Crossbar)

The researchers built a simulation of a crossbar array. Think of this as a giant grid of tiny switches (like a tic-tac-toe board, but much bigger).

The Switches: Each intersection in the grid is a tiny magnetic switch called a Magnetic Tunnel Junction (MTJ).
The Trick: Unlike a normal light switch that is just "on" or "off" (2 states), these switches can be set to 4 different levels of resistance (like a dimmer switch with 4 brightness settings).
The Math: When you send an electrical signal (voltage) into the grid, the switches naturally do the math for you. They multiply the input by their "resistance" setting and add up the results, just like a calculator, but using physics instead of silicon chips.

2. The "Translator" Problem

There is a catch. Computers speak digital (0s and 1s), but this grid speaks analog (smooth, continuous electricity).

The Solution: The team created a "translator" framework. They figured out a simple mathematical recipe to look at the electricity coming out of the grid and convert it back into a digital answer.
The Benefit: This means the grid can work as a standalone "AI coprocessor" without needing to change how the rest of the computer works. It's like plugging a special calculator into your laptop that just works.

3. The Tests: XOR and MNIST

They tested this system with two tasks:

The XOR Puzzle: A simple logic game where the computer has to decide if two inputs are different. The grid solved this perfectly, matching the results of a standard computer.
MNIST (Handwritten Digits): A harder task where the computer identifies numbers written by hand.
- The Result: A standard computer got 97.56% accuracy. The magnetic grid got 94.48%.
- The Gap: The difference wasn't because the grid was "dumb," but because the grid can only hold 4 settings (quantization), while the computer uses infinite precision. It's like trying to paint a detailed portrait using only 4 colors instead of a full palette.

4. The Sources of "Noise"

The researchers looked at what causes errors in this system:

The "Pixelation" Error (Quantization): This is the main culprit. Because the switches only have 4 settings, the math isn't perfectly precise. The more settings you have, the less "pixelated" the answer becomes.
The "Systematic" Error: Imagine if every switch in the grid was slightly bent the same way. This happens if the manufacturing process has a flaw that affects the whole batch. This is very bad for accuracy.
The "Random" Error: Imagine each switch is slightly bent differently due to random heat or tiny manufacturing differences. Surprisingly, this is less harmful. Why? Because when you add up the results from hundreds of switches, the random bends cancel each other out (like a crowd of people walking in slightly different directions still moving forward together).

5. The "Goldilocks" Number of States

The team asked: "How many settings (states) should each switch have?"

Too few: The math is too "pixelated" (high quantization error).
Too many: If you pack too many settings into a tiny switch, the difference between them becomes so small that random noise (heat) blurs them together, making the switch unreliable.
The Finding: There is an optimal number. In a noisy environment, it's actually better to have fewer states (like 4) because they are more distinct and easier to read. In a very quiet, perfect environment, you can have more states for better precision.

Summary

The paper demonstrates that using magnetic switches with 4 distinct states is a powerful way to do AI math. While it's not quite as perfect as a standard computer yet (due to the "pixelation" of having only 4 settings), it gets very close (94% vs 97%).

The key takeaway is that random errors are manageable because they cancel each other out, but systematic errors (flaws affecting everything equally) are dangerous. The researchers also proved that you don't always need the most complex switches; sometimes, a simpler switch with fewer states works better if the environment is a bit "noisy."

This framework gives engineers a blueprint to build future AI chips that are faster and use less energy, by knowing exactly how many "settings" their switches need to be most accurate.

1. Problem Statement

Conventional von Neumann computing architectures face fundamental efficiency limitations, specifically the "von Neumann bottleneck" (data movement between memory and CPU) and the "memory wall" (speed gaps between processors and memory). These issues are exacerbated by the energy and latency demands of modern AI models. While In-Memory Computing (IMC) using crossbar arrays offers a solution by performing analog Matrix-Vector Multiplications (MVM) directly within memory, several challenges remain:

Analog-Digital Interface: Converting digital inputs to analog signals (DAC) and analog outputs back to digital (ADC) introduces significant energy overhead and quantization errors.
Device Non-Idealities: Real-world memristive devices (e.g., MRAM, RRAM) suffer from systematic errors (device-to-device variations, manufacturing defects) and random noise (thermal fluctuations).
Quantization: Mapping full-precision neural network weights to the limited discrete resistance states of hardware cells introduces quantization error, which degrades inference accuracy.
Lack of General Framework: Existing approaches often tightly couple hardware physics with network training, limiting modularity and generalizability to different hardware implementations.

2. Methodology

The authors propose a simulation framework for $N$ -ary crossbar architectures (where cells have more than 2 distinct states) that retrieves MVM results with minimal assumptions about the underlying physics.

Architecture & Device Model:
- The framework simulates a crossbar array using Multi-state Magnetic Tunnel Junctions (M2TJs) with 4 distinct resistance states, based on data from previous studies [29, 30].
- The MVM operation is modeled using Ohm's and Kirchhoff's laws: Input voltages ( $V$ ) multiplied by cell conductances ( $G$ ) generate currents, which are summed to produce output voltages.
- The framework assumes linear encoding of inputs and equidistant resistance levels, allowing the retrieval of the MVM result ( $\tilde{y}$ ) via a linear scaling equation (Eq. 13) applied to the measured output vector. This allows the crossbar to function as a standalone coprocessor without modifying the inference pipeline.
Weight Quantization:
- Full-precision weights are quantized into $N$ equidistant levels ( $A_i$ ) to minimize the squared distance between original weights and quantized values (similar to $k$ -means but with fixed spacing).
- The optimization of the first state ( $A_1$ ) and step size ( $d$ ) is performed using the Powell minimization method.
Error Modeling:
- Systematic Errors: Modeled as a shared random offset ( $\sigma_{NL}$ ) applied to all cells, simulating manufacturing defects (e.g., misalignment of ferromagnetic layers).
- Cell-Specific Errors: Modeled as independent random offsets ( $\sigma_{\perp}$ ) for each cell, simulating thermal noise and device variations.
- Quantization Error: Analyzed by varying the number of states ( $N$ ) from 1 to 64.
Benchmarks:
- XOR Task: A simple 2-layer neural network trained to solve the XOR problem to verify basic non-linear separation capabilities.
- MNIST Task: A standard digit classification task (784 input neurons, 128 hidden, 10 output).
- Dimensionality Reduction: Principal Component Analysis (PCA) was applied to the MNIST input to reduce dimensionality from 784 to 87, simulating the use of smaller crossbar arrays.

3. Key Contributions

Generalized Retrieval Framework: Developed a method to extract MVM results from analog crossbar outputs using only input signals and measured outputs, without requiring the hardware physics to be embedded in the training loop.
Error Source Analysis: Systematically isolated and quantified the impact of quantization error, systematic non-idealities, and random cell-specific noise on inference accuracy.
Averaging Effect Demonstration: Proved that cell-specific random noise is less detrimental than systematic errors of the same magnitude because random errors average out across the array, whereas systematic errors accumulate.
Optimal State Determination: Identified an optimal number of states per cell ( $N_{opt}$ ) that balances quantization error reduction against resistance state resolution loss in noisy environments.
Scaling Laws: Established how MVM error scales with array dimensions, showing that error is constant with the number of rows but scales with the square root of the number of columns.

4. Results

XOR Inference: The simulated crossbar array achieved results equivalent to the full-digital software baseline, confirming that the framework preserves learned model properties.
MNIST Classification:
- Baseline: Software-only accuracy was 97.56%.
- Crossbar (4x4 array, 4 states): Achieved 94.48% accuracy. The 3.08% drop was primarily attributed to weight quantization.
- With PCA: Reducing input dimensionality to 87 features improved the software baseline to 98.03% and the crossbar inference to 95.24%, narrowing the hardware-software gap to 2.79%.
Quantization vs. States ( $N$ ): The Root Mean Squared Error (RMSE) followed a clear $1/N$ trend. Increasing the number of states significantly reduces quantization error, even with small $N$ (e.g., $N=3$ or $4$).
Noise Impact:
- RMSE increased linearly with both systematic ( $\sigma_{NL}$ ) and cell-specific ( $\sigma_{\perp}$ ) error magnitudes.
- However, the slope for cell-specific noise was shallower due to the averaging effect.
Optimal States ( $N_{opt}$ ):
- In low-noise environments, higher $N$ is preferred to minimize quantization error.
- In high-noise environments ( $\sigma_{\perp} > 100 \Omega$ ), $N_{opt}$ decreases because the resolution between states becomes too low to distinguish them reliably.
- Crucially, the system remained robust; even with significant noise, using $N > 4$ states was often beneficial compared to binary implementations.

5. Significance

This work provides a critical bridge between theoretical IMC concepts and practical hardware implementation.

Hardware-Software Co-Design: It demonstrates that specialized algorithms (like PCA) can effectively mitigate hardware limitations (small array size), making current crossbar technology viable for complex tasks.
Design Guidelines: The identification of an optimal number of states ( $N_{opt}$ ) dependent on noise levels offers a concrete design rule for future memristive chip developers. It suggests that in noisy environments, fewer, more robust states may be superior to many fine-grained states.
Robustness of Multibit Systems: The findings validate that multibit ( $N$ -ary) crossbar arrays are superior to binary ones even with imperfect devices, provided the number of states is tuned to the specific noise profile of the hardware.
Modularity: By decoupling the hardware retrieval process from the training phase, the framework facilitates the integration of diverse memristive technologies into standard AI pipelines without retraining the neural networks.

In conclusion, the paper establishes that multibit IMC is a viable path for energy-efficient AI inference, provided that quantization strategies and device noise characteristics are carefully managed through the proposed framework.