⚡ electrical engineering

Variability-Aware Memristor Circuit for Hierarchical Bayesian Temporal Sampling

This paper presents a variabilty-aware memristor circuit that leverages controlled temporal stochasticity and hierarchical training to efficiently implement Bayesian neural networks, achieving a 2.7-fold reduction in memory footprint compared to population-coding approaches while maintaining inference accuracy on the MIT-BIH arrhythmia classification task.

Original authors: Tarcisius Januel, Tifenn Hirtzlin, Olivier Billoint, Djohan Bonnet, Louis Hutin, Damien Querlioz, Jean-Michel Portal, Elisa Vianello

Published 2026-09-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Tarcisius Januel, Tifenn Hirtzlin, Olivier Billoint, Djohan Bonnet, Louis Hutin, Damien Querlioz, Jean-Michel Portal, Elisa Vianello

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Artificial intelligence systems are increasingly being asked not just to make predictions, but to understand how sure they are about those predictions. This shift toward "uncertainty-aware" computing relies on a mathematical approach called Bayesian inference, where the system treats every piece of knowledge as a range of possibilities rather than a single fixed fact. To work, these systems must constantly generate random samples from these ranges to test different outcomes. Doing this efficiently on standard computer chips is difficult because it requires vast amounts of memory and energy. Researchers have long hoped to solve this by using tiny electronic components called memristors, which can store information in a way that naturally mimics the randomness needed for these calculations. However, the physical reality of these components has been a hurdle; the very randomness they possess is often too chaotic and unpredictable to be useful for precise scientific work.

A team of researchers has now demonstrated a new way to harness these components for this specific task, effectively turning a source of error into a reliable tool. They built a custom circuit using a type of memristor made from hafnium oxide, integrated into a standard silicon chip. Instead of trying to force the tiny devices to produce perfect randomness on their own, the team designed a system that separates the storage of information from the generation of randomness. In their setup, three of these tiny devices are grouped together to represent a single piece of knowledge. Two of them store the average value of that knowledge, while the third acts as a control knob for how much the value should fluctuate. Crucially, the actual random fluctuations do not come from the devices themselves. Instead, the circuit injects a steady, controlled stream of external white noise into the system. The third memristor simply adjusts how much of this external noise is allowed to pass through, effectively tuning the level of uncertainty.

The researchers first tested whether the memristors could generate the necessary randomness on their own. They found that the natural fluctuations inside the devices were inconsistent and followed a pattern that changed over time, making them unsuitable for the precise sampling required by the system. The noise was too dependent on the specific history of the device and varied wildly from one component to the next. By introducing the external noise source, they bypassed these limitations entirely. The external noise provided a clean, steady stream of randomness that was the same every time, while the memristors acted only as precise volume controls. This separation allowed the team to program the average value and the amount of uncertainty independently, a feat that was previously difficult to achieve with such compact hardware.

To make this hardware useful for real-world tasks, the team developed a special training method that accounts for the physical quirks of their chip. They treated the inevitable small differences between individual memristors not as errors to be fixed, but as a known part of the system's behavior. By building a model that included these physical variations, they could train a neural network to work perfectly within the constraints of the actual hardware. When they tested this system on a task involving the classification of heart rhythms from a standard medical database, the results were promising. The new approach used significantly less memory than previous methods that relied on large groups of devices to represent uncertainty. It successfully maintained the accuracy of the predictions and the ability to estimate how confident the system was in those predictions. This work shows a clear path toward building compact, energy-efficient chips that can perform complex, uncertainty-aware reasoning, moving a step closer to hardware that can think with the same nuance as the human brain.

Technical Summary: Variability-Aware Memristor Circuit for Hierarchical Bayesian Temporal Sampling

Problem Statement
Bayesian Neural Networks (BNNs) require hardware capable of storing and sampling probabilistic synaptic weights to quantify uncertainty in artificial intelligence. While population coding—using an array of nominally identical devices to exploit device-to-device (D2D) variability—offers a direct representation of uncertainty, it incurs substantial memory overhead, as each synaptic weight requires as many devices as there are samples. Temporal sampling, which uses a single bit-cell repeatedly to generate samples from intrinsic cycle-to-cycle (C2C) or read-to-read (R2R) variability, offers a more compact alternative. However, realizing temporal sampling in hardware is challenging because intrinsic memristor fluctuations often lack the necessary controllability, reproducibility, and spectral properties (specifically, they are often flicker-noise-like rather than white) required for robust Bayesian inference. Furthermore, existing approaches struggle to independently control the mean and standard deviation of the weight distribution without repeated write cycles that degrade device endurance.

Methodology
The authors propose a memristor-based temporal-coding circuit that decouples non-volatile parameter storage from the generation of temporal stochasticity. The methodology comprises three main components:

Circuit Architecture:
- The system utilizes a 2048-memristor demonstrator (64×32 array) fabricated in the back-end-of-line (BEOL) of a 130 nm CMOS technology using TiN/Ti/HfOx/TiN stacks.
- Each synaptic weight is encoded using three memristor devices: one to control the stochastic component ( $G_\sigma$ ) and two differential devices to encode the positive and negative mean contributions ( $G_{\mu+}$ and $G_{\mu-}$ ).
- Instead of relying on intrinsic device noise, the circuit injects an externally generated white-noise source into the $\sigma$ branch. The memristor conductance $G_\sigma$ acts as a programmable gain, controlling the effective RC cutoff frequency and thus the amplitude of the fluctuations reaching the output.
- The mean ( $\mu$ ) is derived from the difference between the currents of the two differential branches, while the standard deviation ( $\sigma$ ) is derived from the fluctuations of the noise-injected branch. The output is processed by a charge trans-impedance amplifier (CTIA) to produce temporal samples following a Gaussian distribution $N(\mu, \sigma^2)$ .
Characterization of Intrinsic Variability:
- The authors experimentally demonstrated that intrinsic memristor variability (D2D, C2C, and R2R) is insufficient for temporal sampling. R2R variability was found to be too small, highly dispersed across devices, and independent of the programmed conductance state.
- Spectral analysis revealed that intrinsic R2R fluctuations follow a $1/f^\beta$ dependence ( $\beta \approx 1$ ), characteristic of flicker noise, rather than the required white noise spectrum ( $\beta = 0$ ).
Hardware-Aware Learning (Hierarchical Variational Models):
- To address the remaining D2D variability in the memristors encoding $\mu$ and $\sigma$ , the authors developed a hierarchical probabilistic model. They treat the experimentally realized weights not as deterministic values but as "stochastic elliptical states" defined by a bivariate Gaussian distribution with mean vector $M$ and covariance matrix $\Sigma$ .
- Training is performed using Hierarchical Variational Models (HVM). A continuous hidden elliptical distribution ( $\epsilon_H$ ) serves as the optimization variable. At each learning step, this continuous state is projected onto the closest experimentally realizable quantized state ( $\epsilon_Q$ ) from a discrete codebook derived from chip measurements.
- The Bayesian loss and gradients are evaluated on the quantized state $\epsilon_Q$ , ensuring the network learns directly against the hardware's physical constraints and uncertainty statistics.

Key Results

Controlled Stochasticity: The proposed architecture successfully establishes a linear, programmable relationship between the memristor conductance $G_\sigma$ and the output standard deviation $\sigma(V_{out})$ when external white noise is injected. Without injection, the relationship is non-monotonic and unreliable.
Spectral Purity: The output noise spectrum with external injection is approximately flat ( $\beta = 0$ ), confirming the generation of white noise, whereas the intrinsic noise of the mean branch remains negligible.
Performance: On the MIT-BIH arrhythmia classification task, the proposed approach achieves a 2.7-fold reduction in memory footprint compared to an equivalent population-coding implementation.
Accuracy and Uncertainty: The hardware-aware training preserves Bayesian inference accuracy and uncertainty estimates. The system demonstrates that a finite number of temporal samples is sufficient to recover most classification and out-of-distribution (OOD) detection performance, offering a controllable trade-off between sampling cost and accuracy.

Significance and Claims
The paper claims to establish a route to compact Bayesian inference using memristive devices by separating the storage of distribution parameters from the generation of temporal randomness. The significance lies in:

Scalability: By using a single cell per weight (with three devices for parameter control) and temporal sampling, the approach avoids the memory overhead of population coding.
Robustness: The explicit modeling of device variability and the use of external noise sources ensure that the system remains robust under hardware non-idealities, avoiding the need for repeated write cycles to shape distributions.
Hardware-Software Co-design: The integration of experimentally measured device statistics into the training loop (via HVM) allows the network to adapt to the specific stochastic nature of the fabricated hardware, rather than attempting to compensate for errors post-training.

The authors conclude that this architecture enables the number of stochastic samples to be increased over time without increasing the non-volatile memory footprint, addressing a key bottleneck in scalable Bayesian hardware.

More like this