💻 computer science

Inverting Neural Networks: New Methods to Generate Neural Network Inputs from Prescribed Outputs

This paper introduces two novel methods, a forward pass root-finding approach and a backward pass layer-wise inversion technique, to generate random-like input images that achieve near-perfect classification scores, thereby revealing vulnerabilities and providing comprehensive coverage of input spaces in neural network inverse problems.

Original authors: Rebecca Pattichis, Sebastian Janampa, Constantinos S. Pattichis, Marios S. Pattichis

Published 2026-03-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Rebecca Pattichis, Sebastian Janampa, Constantinos S. Pattichis, Marios S. Pattichis

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, but slightly mysterious, machine. You put a picture of a cat into the machine, and it shouts, "That's a cat!" You put in a dog, and it says, "That's a dog!"

This machine is a Neural Network. It's great at making these decisions, but it's a "black box." We know what goes in and what comes out, but we don't really know how it makes the connection inside.

The Big Question:
What if we tried to do the opposite? What if we told the machine, "I want you to say 'Cat' with 100% confidence," and asked it to work backward to tell us: What picture would make you say that?

This is what the paper calls the "Inverse Problem." The authors, Rebecca and her team, wanted to see if they could reverse-engineer the machine to find the "perfect" input for any specific output.

Here is how they did it, explained with some everyday analogies:

The Two New Methods

The authors came up with two different ways to solve this puzzle.

1. The "Guess and Check" Method (Forward Pass)

Imagine you are trying to find the exact combination to a safe. You don't know the numbers, but you have a tool that tells you how close you are to the right answer every time you guess.

How it works: The computer starts with a completely random, static-filled picture (like TV snow). It feeds this into the machine. The machine says, "Nope, that looks like a 7, not a 3."
The Trick: The computer uses a mathematical "compass" (called a Jacobian) to see which way to nudge the picture to get closer to "3." It keeps nudging and checking, over and over, until the picture finally tricks the machine into saying "3" with perfect confidence.
The Result: The picture that emerges isn't a real photo of a 3. It looks like a weird, abstract painting that only the machine understands.

2. The "Reverse Assembly Line" Method (Backward Pass)

Imagine a factory assembly line where a raw material goes through several stations to become a finished toy.

Station 1: Adds paint.
Station 2: Adds wheels.
Station 3: Adds a box.

If you have the finished toy and want to know what the raw material looked like, you can't just reverse the process easily because some steps are messy (like mixing paint).

How it works: The authors start with the final answer (the "Cat" label) and work backward through the machine's layers, one by one.
The Secret Sauce: When they hit a step that is hard to reverse (like a linear layer), they realize there are infinite ways to get to that point. So, they add a little bit of "random noise" (like shaking the box) to generate a new version of the input.
The Result: This method creates a whole family of weird, random-looking images that all fool the machine into thinking they are the same class.

The Shocking Discovery: The Machine is "Gullible"

The most important part of the paper is what they found when they looked at the pictures these methods created.

Old Methods: Previous attempts to reverse-engineer these networks created pictures that looked very similar to the real training data (e.g., a blurry but recognizable cat).
New Methods: The authors' new methods created pictures that looked like pure static or random noise. They looked nothing like a cat, a dog, or a number.

The Analogy:
Imagine a security guard at a club.

Old View: We thought the guard only let in people who looked exactly like the VIP list (real cats).
New Discovery: The authors found that you can walk in wearing a clown suit made of static noise, and the guard will still say, "Welcome, VIP!"

Why Does This Matter?

This reveals a vulnerability. It shows that these neural networks aren't actually "learning" what a cat looks like in a human sense. Instead, they are finding weird, mathematical shortcuts that humans can't see.

The Good News: This helps scientists understand how AI thinks. It's like an X-ray for the brain of the machine.
The Bad News: It means these systems can be easily tricked. If a self-driving car thinks a random pattern of static is a "Stop" sign, that's dangerous.
The Future: By understanding these "ghost" images, we can retrain the AI to be smarter and less easily fooled, making our technology safer.

In a nutshell: The authors built a tool to ask AI, "What do you think a cat looks like?" The AI answered with a picture of random static, proving that it sees the world very differently than we do.

1. Problem Statement

Neural networks (NNs) perform complex, high-dimensional mappings from inputs (e.g., images) to outputs (e.g., class probabilities). While forward inference is standard, the inverse problem—determining what specific input images map to a prescribed output distribution (class)—remains challenging.

Goal: To generate input images that yield specific, near-perfect classification scores for a given neural network.
Motivation:
- Explainability: Understanding what features the network relies on for specific classes.
- Security: Identifying vulnerabilities by finding "unrelated" or random-like inputs that fool the network into specific classifications (adversarial examples).
- Data Augmentation: Generating desirable images to retrain networks.
Limitation of Prior Work: Previous methods (e.g., searching training datasets or training decoders) often produced images that were too similar to the original training set, failing to explore the full input space or reveal deep network vulnerabilities.

2. Methodology

The authors propose two distinct algorithms to solve the inverse problem, applicable to both sequential linear networks and modern Transformer architectures.

A. Forward Pass Method (`FORWARDPASSINV`)

This is a general-purpose algorithm applicable to any neural network architecture.

Mechanism: It treats the inversion as a root-finding problem. The goal is to find an input $x$ such that $NN(x) = \text{Target Class Distribution}$ .
Algorithm:
1. Start with a random initial guess for the input image $x_0$ .
2. Compute the Jacobian of the loss function with respect to the input image.
3. Use an accelerated root-finding algorithm (specifically Levenberg-Marquardt) to iteratively update $x$ to minimize the loss between the network output and the target distribution.
Characteristics: Relies on random sampling of the initial guess to converge to different local solutions. It does not require the network layers to be mathematically invertible.

B. Backward Pass Method (`BACKPASSINV`)

This is a layer-by-layer inversion method applicable to networks composed of invertible activation functions (e.g., SELU, LeakyReLU), linear layers, and softmax.

Mechanism: It inverts the network sequentially from the output layer (softmax) back to the input layer.
Key Steps:
1. Softmax Inversion: Inverts the softmax layer using a logarithmic transformation (up to a constant).
2. Activation Inversion: Applies the inverse of the activation function ( $\phi^{-1}$ ). Checks for numerical instability (unbounded values) and returns an error if detected.
3. Linear Layer Inversion: Uses Singular Value Decomposition (SVD) to compute the pseudoinverse of weight matrices ( $W^+$ $W^{+}$ ).
  - Stabilization: To prevent numerical instability, the algorithm zeros out singular values where the condition number ( $\kappa$ ) exceeds a threshold (e.g., 100).
4. Null-Space Sampling (Key Innovation): Since linear layers often map high-dimensional inputs to lower-dimensional outputs, the inverse is not unique. The algorithm generates all possible inputs by adding random vectors sampled from the null-space of the weight matrix ( $W$ $W$ ) to the calculated input.
  - Formula: $x_{i-1} = W_i^+ (x_i - b_i) + \text{Null}(W_i) \cdot S$ , where $S$ is a random vector.
Characteristics: Can generate a distribution of random input samples that satisfy the output constraints, rather than a single solution.

C. Generalized Approach

The paper suggests a hybrid approach: FORWARDPASSINV can invert general blocks, while BACKPASSINV can invert specific blocks (linear + invertible activation). Combining them allows for generating a diverse distribution of random input samples.

3. Key Contributions

Novel Algorithms: Introduction of two distinct inversion methods: a Jacobian-based root-finding approach (Forward) and a layer-wise SVD/Null-space sampling approach (Backward).
No Additional Training: Unlike decoder-based methods, these techniques do not require training new models; they operate directly on pre-trained networks.
Null-Space Exploration: The backward pass explicitly utilizes the null-space of linear layers to generate infinite variations of input images, ensuring comprehensive coverage of the input space.
Architecture Agnostic: The forward pass works on any architecture (including Transformers), while the backward pass handles standard linear/activation blocks.

4. Experimental Results

The authors tested their methods on:

Transformers: Vision Transformers (ViT-tiny and DINOv3-Base) fine-tuned on MNIST.
Sequential Networks: Fully Connected Neural Networks (FCNN) with 1, 2, and 6 linear layers using SELU activations.

Findings:

High Accuracy: Both methods successfully generated inputs that yielded near-perfect classification scores (probability $\ge 0.9$ ) for the target class and low probabilities for others.
Randomness vs. Complexity:
- Generated images from the new methods were random-like and visually distinct from the original MNIST training data.
- As network complexity increased (e.g., from 1-layer to 6-layer FCNNs, or from ViT-tiny to DINOv3), the generated inputs became increasingly random.
- This aligns with the bias-variance tradeoff: complex models have lower bias but higher variance, allowing them to classify highly random noise as specific classes.
Comparison: Previous methods produced images resembling the training set. The new methods produced "ideal" inputs that are far from the training distribution, exposing that the networks rely on features that can be satisfied by random noise.

5. Significance and Conclusion

Vulnerability Exposure: The primary significance is the demonstration that modern neural networks, particularly complex ones like Transformers, are vulnerable to "random-like" inputs. The networks can be forced to classify arbitrary noise as specific classes with high confidence.
Input Space Coverage: The proposed methods provide a more comprehensive understanding of the input space that maps to specific outputs, moving beyond simple interpolation of training data.
Future Work: The authors plan to extend the backward pass method to handle non-invertible activations (like ReLU) using linear programming techniques.

In summary, this paper provides a rigorous mathematical framework for inverting neural networks, revealing that high-performance classifiers may rely on brittle features that allow random noise to be misclassified with high confidence.