💻 computer science

Tensor-Based Batch Fuzzing with Adaptive Perturbation Scaling for Deep Neural Networks

This paper introduces a tensor-based batch fuzzing framework for deep neural networks that leverages adaptive, specification-aware perturbation scaling and batched constraint embedding to achieve significantly higher throughput and violation detection rates compared to traditional sequential approaches.

Original authors: Guanqin Zhang, Yulei Sui

Published 2026-06-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Guanqin Zhang, Yulei Sui

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a quality control inspector for a fleet of self-driving cars. Your job is to find "bugs" in their brains (Deep Neural Networks) by throwing tricky situations at them—like a stop sign that looks a little like a speed limit sign, or a pedestrian wearing a weird outfit. If the car gets confused and makes a mistake, you've found a bug.

The paper introduces a new, super-fast way to do this inspection, called Tensor-Based Batch Fuzzing. Here is how it works, explained through simple analogies.

The Old Way: The Single-File Line

Imagine the old method of testing is like a single cashier at a grocery store.

One customer at a time: The cashier (the computer) takes one customer (one test image), scans their items, checks if they paid correctly, and then moves to the next.
Uniform rules: The cashier applies the exact same rule to every item in the bag. If a bag has a tiny, fragile egg and a heavy brick, the cashier handles them with the same "gentleness" or "force." This isn't efficient; it might crush the egg or be too weak to move the brick.
The bottleneck: Even if the cashier is fast, they can only process one person at a time. If you have 100 customers, it takes 100 times longer than if you could process them all together.

The New Way: The Super-Assembly Line

The authors built a new system that acts like a massive, automated assembly line with a conveyor belt.

Batch Processing (The Conveyor Belt): Instead of one customer, the system puts 100 customers (100 test images) on the belt at once. The computer processes all 100 in a single "sweep." Because modern computer chips are designed to do many things at once (like a choir singing together rather than one person at a time), this is incredibly fast.
Result: The paper claims this new method is 40 times faster than the old single-file method.

The "Adaptive" Magic: Custom-Fitted Gloves

The second big improvement is how the system changes the test images.

The Old Problem: Imagine you are trying to wiggle a key into a lock. If the lock is tight, you need tiny, precise movements. If the lock is loose, you can wiggle it wildly. The old method used the same "wiggle size" for every lock, regardless of how tight or loose it was. This meant it either wasted time on loose locks or couldn't break tight ones.
The New Solution (Adaptive Scaling): The new system looks at each specific lock (each part of the image) and its specific rules (the "specification").
- If a part of the image has a wide, loose rule, the system gives it a big wiggle.
- If a part has a tight, strict rule, the system gives it a tiny, precise wiggle.
The Analogy: It's like a tailor making custom gloves for 100 different hands at once, rather than giving everyone the same size glove. This allows the system to explore the "tricky" areas much more effectively.

How It Works Step-by-Step

Preparation: The system takes a list of rules (specifications) and wraps them around the AI model. Think of this as putting a custom "test harness" around the car's brain.
Selection: It picks a group of test cases (a "batch") that look promising based on past performance.
Mutation (The Wiggle): It applies the "custom wiggle" (adaptive scaling) to all 100 test cases at the same time.
The Test: It runs all 100 through the AI brain in one go.
The Verdict: It instantly checks: "Did any of these 100 cause a mistake?" If yes, it saves that mistake. If no, it keeps the ones that were "interesting" (like activating a new part of the brain) and tries again.

The Results

The authors tested this on three different sets of images (Traffic Signs, Cifar100, and TinyImageNet) using various AI models.

Speed: They found that the new method could process tests 40 times faster than the old way.
Success: In the same amount of time, the new method found 4 times more bugs (mistakes) than the old method.
Why? It's not just that it's faster; it's also smarter. By adjusting the "wiggle size" for each specific part of the image, it finds bugs that the old, "one-size-fits-all" method missed.

Summary

The paper presents a tool that stops testing AI models one by one and starts testing them in massive groups. It also stops using a "one-size-fits-all" approach to changing the test images, instead customizing the changes to fit the specific rules of each test. The result is a system that finds bugs in AI brains much faster and more thoroughly than before.

Technical Summary: Tensor-Based Batch Fuzzing with Adaptive Perturbation Scaling for Deep Neural Networks

Problem Statement

Deep Neural Networks (DNNs) are increasingly deployed in safety-critical domains such as autonomous driving and medical diagnosis. However, their opaque, high-dimensional parameter spaces make systematic reliability assessment difficult. Existing coverage-guided fuzzing frameworks for DNNs, such as DeepHunter and TensorFuzz, suffer from two primary limitations:

Sequential Processing Bottleneck: Traditional fuzzing follows a "one-input-per-iteration" paradigm inherited from classical software fuzzing. This approach fails to leverage the inherent tensorized nature of DNN inference, where hardware-level parallelism allows batches of inputs to be processed with nearly the same kernel-launch overhead as a single input. While some tools (e.g., TensorFuzz) utilize batching for model evaluation, their mutation and feedback mechanisms often remain sequential or NumPy-based, leading to underutilized throughput.
Uniform Perturbation Inefficiency: Most approaches apply a fixed, global perturbation radius ( $\epsilon$ ) across all input dimensions. In practice, input specifications often define non-uniform, per-dimension bounds (e.g., via VNNLib). A uniform step size forces the fuzzer to either under-explore loosely constrained dimensions or over-explore tightly constrained ones, resulting in inefficient search spaces.

Methodology

The authors propose a Specification-Aware Batch Fuzzing framework that shifts the fuzzing paradigm from sequential input-level mutation to batch specification-level execution. The core of the approach involves three key technical components:

1. Wrapped Model Architecture

The framework embeds input constraints and output property checks directly into the neural network as non-trainable layers, creating a "wrapped model" ( $M_{wrapped}$ ).

InputSpecLayer: Encodes per-sample input bounds ( $[l, u]$ ) and enforces feasible region constraints via projection ( $\Pi_{[\Phi]}$ ) during the forward pass.
OutputSpecLayer: Performs vectorized property checks (e.g., top-1 classification robustness) for all samples in a batch simultaneously.
This architecture allows the entire fuzzing loop (seed selection, mutation, inference, coverage tracking, and feedback) to operate on a batch of $B$ specification instances in a single forward pass, treating specifications as first-class tensor dimensions.

2. Adaptive Perturbation Scaling

To address the limitations of fixed perturbation budgets, the authors introduce a scaling mechanism that derives mutation step sizes from the specification-defined feasible ranges using a shared scale factor $s \in (0, 1]$ .

Isotropic Scaling: Computes a global scalar step size $\eta = \bar{r} \cdot s$ , where $\bar{r}$ is the mean feasible range across all dimensions and samples. This broadcasts a uniform perturbation magnitude.
Anisotropic Scaling: Computes a per-dimension step size tensor $S_{b,d} = (u_{b,d} - l_{b,d}) \cdot s$ . This assigns larger steps to dimensions with wider bounds and finer steps to tightly constrained dimensions, enabling consistent exploration under heterogeneous constraints.

3. Tensor-Based Fuzzing Pipeline

The fuzzing process is executed in four phases, all operating on batched tensors of size $B$ :

Seed Selection: Samples $B$ seeds from a corpus based on energy-weighted multinomial selection, prioritizing inputs that previously triggered new coverage or violations.
Mutation: Applies a selected strategy (Gradient/PGD, Boundary, or Random) to the entire batch. The mutation direction is combined with the adaptive scaling factor (scalar or tensor) and projected back onto the feasible region.
Execution: Performs a single batched forward pass through $M_{wrapped}$ . PyTorch hooks capture intermediate activations for coverage tracking, while the OutputSpecLayer flags property violations.
Feedback: Updates per-sample energy scores based on coverage gains and violation status, re-inserting interesting seeds into the corpus.

Key Contributions

The paper summarizes its contributions as follows:

Specification-Aware Batch Framework: A unified data-parallel pipeline that operates at the level of specification instances rather than individual inputs. By embedding constraints into the model, it evaluates $B$ specifications in a single forward pass, significantly improving efficiency.
Adaptive Perturbation Scaling: A scheme that derives mutation magnitudes from specification-defined bounds, supporting both isotropic (global scalar) and anisotropic (per-dimension tensor) scaling to handle heterogeneous input constraints effectively.
Implementation and Evaluation: A PyTorch implementation evaluated on six networks across three benchmarks (TrafficSigns, Cifar100, TinyImageNet) with over 400 specifications.

Experimental Results

The framework was evaluated against a sequential baseline (simulating DeepHunter's one-input-per-iteration paradigm) under a fixed time budget.

Throughput: The tensor-based batch approach achieved up to 40× higher throughput than the sequential baseline. For example, on Cifar100 and TinyImageNet, the speedup was 34–40×, while TrafficSigns saw a 5× improvement (attributed to the lighter network architecture allowing the baseline to run faster).
Violation Discovery: Under the same time budget, the batch approach discovered 4× more violations than the sequential baseline.
- On TrafficSigns, the batch methods found ~75,000 violations compared to ~19,000 for the baseline.
- On Cifar100, the batch methods found ~61,000 violations compared to ~14,000 for the baseline.
Time to First Violation (TTFV): The batch approach identified the first counterexample within 0.06–1.2 seconds, whereas the sequential baseline often approached the 60-second timeout before finding the first violation.
Anisotropic vs. Isotropic: Anisotropic scaling showed particular advantages on benchmarks with heterogeneous constraints (e.g., TinyImageNet) and at smaller scale factors ( $s$ ), outperforming isotropic scaling by up to 38% in specific configurations.
Batch Size Impact: Statistical analysis (Mann-Whitney U test) confirmed that increasing batch size $B$ significantly improves violation yield, though per-instance refinement decreases as $B$ grows, indicating a trade-off between throughput and refinement.

Significance and Claims

The paper claims that its primary significance lies in overcoming the inherent limitations of sequential and uniform perturbation approaches in DNN fuzzing. By lifting the fuzzing loop to the tensor level and integrating specification constraints directly into the model execution, the framework achieves substantial improvements in both testing throughput and the discovery of counterexamples.

The authors emphasize that while the sequential paradigm can eventually discover violations given a budget scaled by the batch size ( $B \times 60s$ ), its one-input-per-iteration design makes counterexample generation significantly slower in wall-clock time. The proposed method demonstrates that treating specifications as first-class tensor dimensions allows for the simultaneous processing of multiple mutation chains, leading to a "4× more violations" gain within the same real-world time window. The work positions itself as a complementary approach to formal verification, providing concrete counterexamples quickly rather than exhaustive safety certificates.