🤖 machine learning

Formally Verifying Analog Neural Networks Under Process Variations Using Polynomial Zonotopes

This paper introduces a formal verification approach for analog neural networks under process variations using polynomial zonotopes and a polynomial-based model, which significantly reduces verification time from days to seconds while accurately enclosing 99% of variation samples without relying on Monte Carlo simulations.

Original authors: Yasmine Abu-Haeyeh, Tobias Ladner, Matthias Althoff, Lars Hedrich

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yasmine Abu-Haeyeh, Tobias Ladner, Matthias Althoff, Lars Hedrich

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are building a super-fast, energy-efficient brain using tiny electronic circuits instead of software code. This is the promise of Analog Neural Networks. Unlike the digital computers in your phone that use strict "on/off" switches (0s and 1s), these analog circuits use smooth, flowing electrical currents to mimic how real neurons work. They are incredibly fast and use very little battery power, making them perfect for wearable devices and sensors.

However, there's a catch. Because these are physical circuits built from sand (silicon), they are incredibly sensitive to the manufacturing process. Think of it like baking a cake: even if you follow the recipe perfectly, tiny variations in the oven temperature, the humidity, or the exact size of your measuring spoon can change the outcome. In chip manufacturing, these "variations" mean that no two transistors are exactly identical. One might be slightly wider, another slightly thicker.

For a digital computer, a tiny difference doesn't matter; a 1 is still a 1. But for an analog brain, a tiny difference can cause the whole system to make a wrong guess.

The Problem: The "Monte Carlo" Trap

To make sure these analog brains work correctly despite these manufacturing flaws, engineers usually use a method called Monte Carlo simulation.

Imagine you want to know if a new bridge will hold up in a storm. The traditional way is to build a model, then simulate a storm 1,000 times with slightly different wind speeds and rain intensities. You do this over and over again to see if the bridge ever collapses.

The Paper's Claim: Doing this for analog neural networks is like trying to simulate a storm 1,000 times for every single layer of a complex brain. It takes days of computing time. For a simple image recognition task (like identifying handwritten numbers), the paper notes it took the researchers nearly 29,000 seconds (about 8 hours) just to run these simulations on a standard computer.

The Solution: The "Polynomial Zonotope"

The authors of this paper propose a smarter way. Instead of simulating the storm 1,000 times, they create a mathematical map that covers all possible storms at once.

Here is how they did it, broken down into simple steps:

1. Finding the "Main Culprits"
First, they looked at the circuit and realized that while there are dozens of variables that could go wrong (like the thickness of the wire or the voltage), only two specific variables actually caused the most trouble. It's like realizing that in a car, only the tire pressure and the engine temperature really matter for safety; the color of the seats doesn't. They focused their attention on these two "dominant" factors.

2. Creating a "Shape" for the Errors
Instead of checking individual points, they used a mathematical shape called a Polynomial Zonotope.

The Analogy: Imagine you are trying to catch a flock of birds that are flying in a messy, unpredictable pattern.
- The Old Way (Monte Carlo): You try to catch each bird one by one. It takes forever, and you might miss a few.
- The New Way (Polynomial Zonotopes): You stretch a giant, flexible, transparent net over the entire flock. This net is shaped perfectly to wrap around the birds' movement, accounting for how the wind (the manufacturing variations) pushes them.
- This "net" is the Polynomial Zonotope. It is a special kind of shape that can bend and twist to fit the complex, curved paths the errors take, which simpler shapes (like straight boxes) cannot do.

3. The "Verification" Test
Once they have this net, they don't need to simulate the circuit thousands of times. They just check: "Does the net stay inside the 'safe zone'?"

If the net (representing all possible outcomes) never touches the "unsafe zone" (where the brain makes a mistake), then the circuit is guaranteed to be safe, no matter how the manufacturing varied.

The Results: From Days to Seconds

The paper tested this method on three different tasks:

Breast Cancer Detection: Distinguishing between benign and malignant tumors.
Iris Classification: Identifying different types of flowers.
MNIST: Recognizing handwritten digits (0-9).

The Outcome:

Speed: The old method took hours or even days. The new method took seconds.
- For the handwritten digits (MNIST), the old way took ~8 hours. The new way took 1.77 seconds.
Accuracy: The new method was incredibly precise. It successfully "caught" 99% of all the possible variations. In other words, if you built 100 of these chips, the mathematical net guaranteed that 99 of them would work correctly.

Why This Matters (According to the Paper)

The authors emphasize that this isn't just about being faster; it's about feasibility.

Scalability: Because the method is so fast, engineers can now design larger, more complex analog brains without waiting weeks for verification.
Robustness: It allows designers to see exactly how much variation a chip can handle before it fails. For example, they found that while chips with small variations (1 or 2 standard deviations) still worked well, chips with huge variations (3 standard deviations) started to fail. This helps designers know where to draw the line.

Summary

In short, the paper presents a new "mathematical net" (Polynomial Zonotopes) that allows engineers to verify the safety of analog neural networks in seconds instead of days. By focusing on the most important manufacturing flaws and wrapping them in a flexible mathematical shape, they proved that these energy-efficient circuits can be trusted to work correctly, even when built with imperfect materials.

Technical Summary: Formal Verification of Analog Neural Networks Under Process Variations Using Polynomial Zonotopes

Problem Statement

Analog neural networks (ANNs) offer significant advantages in power consumption, processing speed, and chip area compared to digital counterparts, making them ideal for energy-constrained Internet of Things (IoT) applications. However, as physical circuits, ANNs are highly susceptible to manufacturing process variations (e.g., gate width, threshold voltage, oxide thickness). These variations cause deviations from the nominal model, potentially compromising the safety and accuracy of the network.

Traditional verification methods rely on time-consuming Monte Carlo simulations to validate circuit functionality under these variations. For complex networks, such as Convolutional Neural Networks (CNNs), these simulations can take days to complete, creating a bottleneck in the design cycle. Furthermore, existing modeling approaches often fail to capture the full system-level behavior or lack scalability across different network architectures.

Methodology

The authors propose a formal verification framework that replaces Monte Carlo simulations with set-based reachability analysis using polynomial zonotopes. The methodology consists of three main stages:

1. Sensitivity Analysis and Parameter Reduction

The authors analyzed a 130 nm CMOS analog neuron circuit using the PSP transistor model. Through sensitivity analysis of over 50 process parameters, they identified that only two dominant parameters ( $\phi_1$ and $\phi_2$ ), which influence the surface potential of NMOS transistors, significantly impact circuit performance. Other parameters were found to have negligible effects or were mitigated by the circuit topology (e.g., current mirrors).

2. Polynomial-Based Parametric Modeling

Instead of simulating the full circuit physics for every variation, the authors derived a polynomial model to abstract the neuron's behavior.

Weight Modeling: The weights ( $\tilde{W}_k$ ) are modeled as third-order polynomials of the dominant parameters $\phi_1$ and $\phi_2$ .
Bias and Noise: Bias terms are treated as negligible, while the remaining minor variations are captured by a single third noise symbol ( $d_3$ ).
Formulation: The network behavior is expressed as a function of these parameters, allowing the weights to be represented as sets rather than fixed values. Specifically, the weights are modeled as:
$\tilde{W}_k = \sum_{i=0}^{3} \sum_{j=0}^{3} A_{k,i,j} \phi_1^i \phi_2^j + d_3$
where coefficients are determined via polynomial regression on Monte Carlo data for each quantized weight.

3. Reachability Analysis with Polynomial Zonotopes

The verification process propagates input sets through the network using polynomial zonotopes, a set representation capable of handling dependencies between variables.

Set Propagation: The input set is propagated layer-by-layer. Linear layers involve matrix multiplication with the polynomial weight sets.
Non-linearity Handling: ReLU activation functions are enclosed using standard techniques for set-based computing.
Verification: The final output set is enclosed. If this enclosure does not intersect with the set of "unsafe" outputs (e.g., incorrect classification boundaries), the network is formally verified as safe for the specified variation range (e.g., $\pm 3\sigma$ ).

Key Contributions

Optimized Polynomial Model: A parametric model that precisely abstracts analog neuron performance under process variations using only two dominant parameters and a noise term, reducing complexity while maintaining accuracy.
System-Level Formal Verification: The application of reachability analysis with polynomial zonotopes to verify circuit-level models under process variations, avoiding the need for exhaustive Monte Carlo simulations.
Scalability: The approach is demonstrated to be scalable across different architectures, including fully-connected and convolutional neural networks (CNNs).
Efficiency: The method drastically reduces verification time while enclosing a high percentage of variation samples.

Experimental Results

The approach was evaluated on three datasets: Wisconsin Breast Cancer, Iris, and MNIST (both fully-connected and CNN architectures).

Verification Time: The proposed method reduced verification time from days to seconds.
- MNIST CNN: Cadence Monte Carlo simulation took ~29,592 seconds (approx. 8.2 hours); the proposed approach took 3.37 seconds.
- Iris: Simulation took ~865 seconds; the proposed approach took 1.15 seconds.
Enclosure Accuracy: The polynomial zonotope enclosure successfully captured approximately 99% of the Monte Carlo variation samples across all datasets (ranging from 97.77% to 99.51%).
Model Necessity: An ablation study comparing polynomial zonotopes against standard zonotopes (linear relaxations) demonstrated that linear models were insufficient. Standard zonotopes produced enclosures that crossed decision boundaries, failing to verify safety, whereas polynomial zonotopes provided tight enclosures that successfully verified the specifications.

Significance and Claims

The paper claims that its primary significance lies in enabling the formal verification of analog neural networks under process variations in a computationally feasible timeframe. By shifting from statistical simulation (Monte Carlo) to deterministic set-based analysis, the authors enable:

Rapid Design Cycles: Designers can quickly assess the robustness of analog circuits against manufacturing variations without waiting for days of simulation.
Safety Guarantees: The method provides formal guarantees that the network will function correctly within a specified variation range (e.g., 99% of samples), which is critical for safety-critical IoT applications.
Robustness Assessment: The approach allows for the quantification of how many manufactured circuits might be faulty (e.g., identifying that circuits deviating beyond $2\sigma$ may lose accuracy), guiding the need for more robust circuit designs.

The authors note that while the current models are relatively small, this is a limitation of the physical robustness of analog circuits rather than the verification method itself. They conclude that this approach can significantly speed up the design cycle for larger, more robust analog inference circuits.