🤖 machine learning

Adversarial Vulnerabilities in Neural Operator Digital Twins: Gradient-Free Attacks on Nuclear Thermal-Hydraulic Surrogates

This study reveals that neural operator digital twins for nuclear thermal-hydraulic systems are critically vulnerable to undetectable, gradient-free adversarial attacks using fewer than 1% of inputs, which can induce catastrophic prediction errors and highlights the need for robustness guarantees beyond standard validation before deployment.

Original authors: Samrendra Roy, Kazuma Kobayashi, Souvik Chakraborty, Rizwan-uddin, Syed Bahauddin Alam

Published 2026-03-25

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Samrendra Roy, Kazuma Kobayashi, Souvik Chakraborty, Rizwan-uddin, Syed Bahauddin Alam

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Digital Twin" Trap

Imagine you have a Digital Twin of a nuclear power plant. It's a super-smart computer program that acts like a crystal ball. Instead of waiting hours for a real physical simulation, this twin can instantly predict what's happening inside the reactor (temperature, pressure, flow) just by looking at a few sensor readings.

Scientists are excited because these "Neural Operators" are incredibly fast and accurate. They are like genius chefs who can taste a single drop of soup and instantly tell you exactly how the whole pot tastes, down to the last grain of salt.

The Problem: This paper discovered that these genius chefs have a terrifying blind spot. They are so sensitive that if you whisper a tiny, almost invisible lie into their ear, they might scream that the reactor is safe when it's actually about to melt down.

The Attack: The "Whisper" vs. The "Scream"

The researchers tested four different types of these AI models. They tried to trick them using Adversarial Attacks.

The Analogy: Imagine you are trying to fool a security guard.
- The Old Way (Gradient Attacks): You try to push the guard hard, but he sees you coming and blocks you. This requires you to know his secrets (how his brain works).
- The New Way (This Paper): You use a gradient-free method. You don't need to know his secrets. You just try different tiny nudges until you find the one specific spot where a tiny nudge makes him collapse.

The Shocking Result:
The researchers found that they could change less than 1% of the data the AI was looking at (like changing the reading on just one or two sensors out of 100).

The Input: A tiny change. The new sensor reading is still within normal, safe limits. It looks perfectly real to any human or standard alarm system.
The Output: The AI's prediction goes completely crazy. The error jumps from a tiny 1.5% (perfectly fine) to 60% (catastrophic failure).
The Stealth: The AI's new, wrong prediction still looks "normal" to standard safety checks. It passes the Z-score test (a statistical check for anomalies). It's like a wolf in sheep's clothing that the shepherd's dog can't smell.

The Four Models: A Tale of Four Architects

The paper tested four different "architectures" (designs) of these AI models. They reacted very differently to the attacks, which the authors explain using a "Sensitivity Map."

Think of the AI's brain as a building with 102 doors (inputs). The researchers wanted to know: Which door, if you pushed it, would make the whole building shake?

They introduced a concept called Effective Perturbation Dimension ( $d_{eff}$ ). Think of this as a "Focus Score."

Low Focus Score ( $d_{eff} \approx 1$ ): The building is held up by one single pillar. If you push that one pillar, the whole thing falls.
High Focus Score ( $d_{eff} \approx 30$ ): The building is supported by 30 pillars. You have to push 30 of them at once to make it fall.

Here is how the four models fared:

1. POD-DeepONet: The "Fragile but Capped" Model

The Vibe: This model is like a house built on one single, super-strong pillar.
The Attack: It is extremely easy to find that one pillar. The researchers found it immediately.
The Twist: Even though they found the pillar, the house didn't collapse completely. Why? Because this model has a "safety net" (Low-Rank Projection). It forces all predictions to stay within a smooth, limited range. It's like a trampoline: you can jump high, but you can't fly into space.
Result: It was easy to attack, but the damage was limited.

2. S-DeepONet: The "Perfect Storm" (Most Dangerous)

The Vibe: This model is like a house with 4 or 5 main pillars, and they are all made of glass.
The Attack: The researchers found these 4-5 pillars easily. Because the pillars are glass (high sensitivity), pushing them causes massive damage.
Result: This was the worst-case scenario. A tiny push on 3 sensors caused the AI to predict a reactor meltdown when everything was fine. It had the perfect mix of "easy to find" and "easy to break."

3. MIMONet & NOMAD: The "Fortress"

The Vibe: These models are like a fortress with 30 sturdy pillars spread all over.
The Attack: To break this, the attacker has to push 30 different sensors at the exact same time.
Result: It's much harder to break. The researchers had to work much harder (change more sensors) to get the same level of chaos.

The "Two-Factor" Rule of Danger

The paper concludes with a simple rule for how dangerous a model is. It's not just about how sensitive it is; it's about two things multiplied together:

Concentration: How many inputs does it take to break the model? (Fewer is worse).
Amplification: How much does the model blow up when you break it? (More is worse).

S-DeepONet was the most dangerous because it had low concentration (easy to find the weak spot) AND high amplification (the weak spot caused a huge explosion).

Why This Matters for the Real World

We are moving toward a future where AI controls nuclear plants, power grids, and weather prediction. We currently trust these models because they are "accurate" on paper.

The Paper's Warning:

Accuracy is not Safety: A model can be 99% accurate on normal days and 0% reliable on a day when someone tries to trick it.
The Invisible Threat: Because these attacks are so small (changing less than 1% of data) and stay within physical limits, current safety alarms won't see them. The AI will confidently tell the operator, "Everything is fine," while the reactor is actually in danger.
The Solution: We can't just build faster models; we have to build robust models. We need to design them so they don't have those "glass pillars." We need to test them with these "whisper" attacks before we let them control real-world infrastructure.

Summary Metaphor

Imagine a Digital Twin is a weather forecaster.

Normal Day: The forecaster is perfect.
The Attack: A hacker changes the temperature reading on one sensor in a remote village by a tiny amount (0.1 degrees).
The Result: The forecaster, due to a hidden flaw in their logic, suddenly predicts a Category 5 Hurricane hitting the city, even though the sky is blue.
The Danger: The city evacuates, causing panic and economic ruin, all because of a tiny, invisible lie in one sensor.

This paper proves that our current "genius" AI weather forecasters (and nuclear twins) are vulnerable to exactly this kind of trick, and we need to fix their logic before we let them run the show.

1. Problem Statement

Neural operators (e.g., DeepONet, FNO) are rapidly becoming the predictive core of digital twins for safety-critical infrastructure, such as nuclear reactors, enabling real-time reconstruction of physical fields from sparse sensor data. While these models achieve high accuracy (often <2% error) on validation datasets, their adversarial robustness remains uncharacterized.

The paper addresses a critical gap: Can neural operators be fooled by physically plausible, sparse perturbations that remain undetectable by standard validation metrics?

The Threat Model: An adversary corrupts sensor readings (e.g., inlet velocity, temperature, or wall heat flux) before they reach the digital twin.
Constraints: Perturbations must be physically feasible (within sensor calibration ranges and conservation laws) and sparse (affecting <1% of inputs) to avoid triggering anomaly detection.
The Risk: A successful attack could cause the digital twin to report safe thermal margins while actual conditions approach material failure limits, leading to catastrophic operational decisions.

2. Methodology

The authors propose a gradient-free adversarial attack framework tailored for high-dimensional continuous field regression, moving beyond standard classification-based attacks.

A. Attack Framework: Differential Evolution (DE)

Since neural operators in industrial settings are often proprietary black boxes (lacking gradient access) and have high-dimensional outputs ( $\mathbb{R}^{15,908}$ ), gradient-based methods (like PGD) are often infeasible or inefficient.

Algorithm: The authors adapt Differential Evolution (DE), a population-based optimization algorithm.
Encoding: Each candidate solution encodes $k$ pairs of (index, value), representing which input coordinates to perturb and by how much.
Constraints:
- Sparsity ( $L_0$ ): Limits the number of perturbed inputs ( $k \in \{1, 3, 5, 10\}$ ).
- Physical Feasibility: Perturbations are constrained to $\pm 1\sigma$ of the training distribution in standardized space, ensuring they map to realizable physical operating conditions (e.g., valid temperatures and velocities).
Objective: Maximize the relative $L_2$ error between the clean prediction and the attacked prediction.

B. Benchmark Setup

Domain: Nuclear thermal-hydraulics (single-pass rectangular-channel heat exchanger).
Inputs: 102-dimensional boundary conditions (2 global parameters: inlet velocity/temperature; 100 spatial points for wall heat flux).
Outputs: 3,977 mesh points $\times$ 4 fields (Pressure, $u, v, w$ velocities).
Models Evaluated: Four distinct neural operator architectures:
1. MIMONet: Concatenation-based mapping.
2. NOMAD: Multi-branch with multiplicative fusion.
3. S-DeepONet: Sequential DeepONet using a GRU encoder for boundary dependencies.
4. POD-DeepONet: Uses Proper Orthogonal Decomposition (POD) for low-rank output projection.

C. Diagnostic Metric: Effective Perturbation Dimension ( $d_{eff}$ )

To explain why some models are more vulnerable than others, the authors introduce $d_{eff}$ , a Jacobian-based metric quantifying the concentration of input-output sensitivity:
$d_{eff} = \frac{(\sum_{i=1}^d \|J_{b}[:, i]\|_2)^2}{\sum_{i=1}^d \|J_{b}[:, i]\|_2^2}$

$d_{eff} \approx 1$ : Sensitivity concentrated in a single direction (highly vulnerable to single-point attacks).
$d_{eff} \approx d$ : Sensitivity uniformly distributed (robust to sparse attacks).

3. Key Contributions

Discovery of Extreme Vulnerability: Demonstrated that neural operators are acutely vulnerable to extremely sparse ( $L_0 < 1\%$ ) and physically plausible perturbations. Attacks can degrade validated accuracy from $\sim1.5\%$ to 37–63% relative $L_2$ error.
Gradient-Free Efficiency: Showed that DE outperforms gradient-based attacks (PGD) on architectures with gradient pathologies (e.g., POD-DeepONet) and is essential for black-box scenarios.
Two-Factor Vulnerability Model: Established that attack success depends on the interplay of:
- Sensitivity Concentration ( $d_{eff}$ ): How localized the sensitivity is.
- Sensitivity Magnitude: The absolute strength of the input-output coupling.
- Key Insight: High concentration alone (POD-DeepONet) does not guarantee high vulnerability if the output projection limits error amplification.
Stealth Demonstration: Proved that single-point attacks ( $L_0=1$ ) are 100% invisible to standard z-score anomaly detection, as the perturbations remain within statistical norms of normal operation.

4. Key Results

A. Attack Success Rates

S-DeepONet: The most vulnerable architecture. Achieved 94.5% success with just 3 perturbed inputs ( $L_0=3$ ) and 100% with 5 inputs at a 30% error threshold.
POD-DeepONet: Despite having the most extreme sensitivity concentration ( $d_{eff} \approx 1$ ), it showed moderate vulnerability (max ~30% success). This is due to its low-rank output projection acting as a "ceiling" on error amplification.
MIMONet & NOMAD: Exhibited distributed sensitivity ( $d_{eff} \approx 32$ ), requiring coordinated multi-point attacks to achieve high success rates, but still vulnerable (up to 62% success for NOMAD at $L_0=10$ ).

B. Field-Level Impact

Attacks caused spatial hotspot relocations, flow pattern inversions, and safety margin violations.
Crucially, while global error metrics (mean $L_2$ ) spiked, the attacks remained undetectable by aggregate validation metrics.
Amplification: A perturbation affecting <1% of inputs resulted in a 5–8x larger relative change in the output field.

C. Theoretical Findings

Impossibility Result: The paper proves that for neural operators achieving high Sobolev accuracy, adversarial vulnerability is an intrinsic consequence of faithful operator approximation. As approximation error ( $\epsilon$ ) decreases, the gap between clean and adversarial error diverges.
Transferability: Adversarial examples are highly architecture-specific. Perturbations crafted for S-DeepONet failed to fool other models (transfer success < 6%), suggesting ensemble methods could be an effective defense.

5. Significance and Implications

Safety-Critical Deployment: The findings challenge the assumption that models with high validation accuracy are trustworthy for nuclear or grid management. Standard validation protocols are insufficient for assessing adversarial robustness.
New Attack Surface: The paper identifies the "sensor-to-prediction" pathway in digital twins as a novel and dangerous attack surface, distinct from traditional actuator manipulation.
Design Guidelines:
- Pre-deployment Diagnostics: Engineers should evaluate both $d_{eff}$ and sensitivity magnitude before deployment. Models with low $d_{eff}$ and high magnitude are highest risk.
- Defense Strategies: Proposed defenses include architectural ensembles (to exploit low transferability), physics-informed anomaly detection (checking conservation laws), and sensitivity regularization during training.
Broader Impact: This work establishes that the gap between interpolation accuracy and adversarial robustness is a fundamental challenge for scientific machine learning, requiring new certification standards for AI in cyber-physical systems.

In conclusion, the paper provides a rigorous, physics-aware stress-testing framework for neural operators, revealing that current state-of-the-art models harbor structural vulnerabilities that could be exploited to cause catastrophic failures in safety-critical infrastructure while remaining invisible to conventional monitoring systems.