Induced Numerical Instability: Hidden Costs in Multimodal Large Language Models

This paper introduces a novel attack method that induces numerical instability in multimodal large language models by optimizing a specific loss function to generate images, causing significant performance degradation across state-of-the-art models and datasets that is distinct from traditional adversarial perturbations.

The Gist

Here is an explanation of the paper "Induced Numerical Instability: Hidden Costs in Multimodal Large Language Models" using simple language and creative analogies.

The Big Idea: The "Whispering Ghost" Attack

Imagine you have a super-smart robot assistant (a Multimodal Large Language Model, or LVLM) that can look at a picture and answer questions about it. You might think the only way to trick this robot is to show it a picture of a cat that looks like a dog, or to draw a weird scribble on the photo that confuses its eyes.

This paper discovers a brand new way to break the robot. Instead of changing what the robot sees, the researchers change how the robot thinks.

They found that by making tiny, invisible tweaks to the numbers inside the computer's brain, they can cause the robot to hallucinate wildly, giving answers that make no sense, even though the picture looks exactly the same to a human.

---

The Analogy: The "Fuzzy Calculator"

To understand this, imagine the robot's brain is a massive team of accountants doing math.

1. The Short-Cut (Half-Precision): To save time and memory, these accountants don't use infinite precision. They round off numbers. Instead of saying "3.14159265...", they say "3.14". This is called Half-Precision. It's like using a ruler with only big markings instead of tiny millimeter lines. Usually, this is fine.
2. The Rounding Error: Sometimes, when you add up thousands of these rounded numbers, the tiny errors stack up. It's like if you round every step of a long journey; by the end, you might be miles off course.
3. The Attack: The researchers realized that if they could nudge the starting numbers just the right way, they could force the accountants to make the worst possible rounding mistakes. They aren't changing the image; they are changing the math behind the image.

The "Domino Effect"

The paper describes two levels of this instability:

• Level 1: The Ruler (Implementation Level): This is the rounding error mentioned above. It's like using a slightly bent ruler.
• Level 2: The Amplifier (Functional Level): This is where it gets scary. The robot's brain is designed so that small changes can get blown up into huge changes.
  • Analogy: Imagine a microphone that is slightly too sensitive. If you whisper a tiny "hello" into it, it might feedback and scream. The researchers found a way to whisper a specific "hello" (a tiny pixel change) that causes the robot to scream nonsense.

What Happened in the Experiments?

The researchers tested this on several famous AI models (like LLaVA and Idefics) using standard datasets (like Flickr30k and VQAv2).

The Results were shocking:

• The Input: They took a picture of a girl sunbathing with a purple towel. To a human, the "attacked" picture looked identical to the original.
• The Output (Clean): The AI correctly said, "A woman wearing a purple scarf lays on a wooden surface."
• The Output (Attacked): The AI looked at the same picture and said, "The purple shirt man is fighting with the other man."

They did this with many questions:

• Question: "What town is this?" -> Answer: "Burnaby."
• Attacked Answer: "Newark." (Completely wrong city).
• Question: "What is on the plate?" -> Answer: "Cake."
• Attacked Answer: "A steak with veggies."

Why is this different from "Adversarial Attacks"?

Usually, when we talk about "hacking" AI, we think of Adversarial Attacks.

• Adversarial Attack: Like putting a sticker on a stop sign that makes a self-driving car think it's a speed limit sign. You are changing the visual pattern to trick the AI's pattern recognition.
• Numerical Instability (This Paper): Like whispering a specific frequency into a speaker that makes the amplifier blow a fuse. You aren't changing the picture's pattern; you are exploiting the math engine inside the computer. The AI isn't "confused" by the image; its internal math is just breaking down.

The "Hidden Cost"

The paper calls this a "Hidden Cost" because:

1. It's Invisible: You can't see the attack. The image looks perfect.
2. It's Universal: It works on different models, different sizes, and different tasks.
3. It's Hard to Fix: You can't just "train" the AI to ignore it easily, because the problem isn't in the training data; it's in the fundamental way computers do math (floating-point arithmetic).

The Takeaway

This research is a wake-up call. We are building AI systems that are incredibly powerful, but they are running on a foundation of "fuzzy math" (half-precision) to save money and speed.

The authors are saying: "We found a way to make these powerful robots hallucinate just by tweaking the math, not the picture. This is a new kind of weakness we need to understand and fix before we let these robots drive cars or manage hospitals."

It's like discovering that a super-strong bridge doesn't collapse because of heavy trucks, but because of a specific, tiny vibration that makes the steel vibrate until it snaps.

Technical Summary

Here is a detailed technical summary of the paper "Induced Numerical Instability: Hidden Costs in Multimodal Large Language Models."

1. Problem Statement

The paper addresses a novel and previously under-explored failure mode in Multimodal Large Language Models (MLLMs), specifically Large Vision-Language Models (LVLMs). While traditional adversarial attacks focus on maximizing task-specific loss (e.g., misclassification) or introducing perceptible noise, this work investigates numerical instability as a distinct vector of degradation.

• The Core Issue: Modern LVLMs often utilize half-precision (float16) arithmetic to optimize memory usage and inference speed. This introduces rounding errors inherent to the IEEE 754 floating-point standard.
• The Hypothesis: The authors posit that by carefully modifying input images, one can induce a cascade of rounding errors and amplify local numerical sensitivities within the model's computational graph. This leads to significant performance degradation in downstream tasks (captioning, VQA) even when the input perturbation is imperceptible to humans and does not follow traditional adversarial patterns.
• Distinction: Unlike standard adversarial perturbations that target the semantic decision boundary, this approach targets the implementation-level precision and functional sensitivity of the model, causing errors that cannot be resolved simply by increasing data type precision (e.g., switching to float32) because the instability is also rooted in the model's internal function dynamics.

2. Methodology

The authors propose a white-box, gradient-based optimization framework to construct "numerically unstable" inputs.

A. Problem Formulation

The goal is to find a perturbation $\delta$ (bounded by $\epsilon$) added to an input image $X_I$ such that the numerical error in the model $M$ is maximized. The input is $X' = (X_I + \delta, X_P)$, where $X_P$ is the text prompt.

B. The Proxy Loss Function

Directly calculating the infinite-precision ground truth for every operation to measure error is computationally intractable. Instead, the authors derive a proxy loss based on Lemma 3.1 (Forward error bound under floating-point rounding).

• Theoretical Basis: The lemma states that the absolute forward error is bounded by terms proportional to the magnitude of the input ($\|x\|$) and the magnitude of the output ($\|f(x)\|$).
• Optimization Objective: To maximize numerical error, the authors propose maximizing the magnitude of intermediate activations throughout the network. The proxy loss is defined as:
  $$\mathcal{L}_{proxy} = \sum_{k \in [1, K]} |\hat{\theta}_k(X_I + \delta)_D|$$
  Where $\hat{\theta}_k$ represents the output of the $k$-th elementary operation in the computational graph using limited precision $D$.
• Mechanism: By forcing intermediate values to be large, the rounding errors (which scale with magnitude) are amplified. Furthermore, maximizing these magnitudes implicitly drives large changes in the output of preceding functions, exploiting the model's functional sensitivity (local Lipschitz constants).

C. Implementation Details

Optimizing this loss presents challenges, such as vanishing gradients and precision loss during the optimization process itself. The authors employ specific techniques:

1. Mixed Precision: The perturbation $\delta$ and accumulated loss are stored in float64 to prevent the optimizer from losing precision during updates, while the model forward pass remains in float16.
2. Gradient Scaling: To handle small gradient magnitudes, the update rule mimics the Fast Gradient Sign Method (FGSM) but uses the sign of the gradient rather than the raw value:
   $$\delta'_{i+1} = \delta'_i + \alpha \cdot \text{sign}(\nabla_{\delta'} \mathcal{L}_{proxy})$$
   This ensures stable updates even when gradients are numerically tiny.

3. Key Contributions

1. Novel Failure Mode Identification: The paper identifies "Induced Numerical Instability" as a distinct failure vector orthogonal to traditional adversarial attacks. It demonstrates that models can be broken not by semantic manipulation, but by exploiting floating-point arithmetic limitations.
2. Proxy Loss Framework: The authors introduce a computationally efficient, gradient-based method to induce these instabilities without requiring ground-truth labels or task-specific loss functions.
3. Comprehensive Evaluation: The method is validated across state-of-the-art LVLMs of varying sizes and architectures (LLaVA-v1.5-7B, Idefics3-8B, SmolVLM-2B, Janus-Pro-1B) and standard benchmarks (MSCOCO, Flickr30k, TextVQA, VQAv2, POPE).
4. Analysis of Precision Sensitivity: The study reveals that while increasing precision (float16 $\to$ float32) offers marginal improvement, it does not fully mitigate the degradation, suggesting the vulnerability stems from both precision limits and functional amplification within the network.

4. Experimental Results

The experiments show that the proposed method (NUM) causes significantly worse performance degradation compared to baselines like random noise (RAND), Gaussian noise (GAUS), and no noise (NONE).

• Performance Drop:
  • On Idefics3-8B with the MSCOCO dataset, the CIDEr-D score dropped from 0.664 (clean) to 0.273 (NUM), a ~59% degradation.
  • In contrast, Gaussian noise baselines showed minimal degradation (e.g., dropping only ~5% in VQAv2 for SmolVLM2).
• Semantic Drift:
  • Qualitative examples (Figure 4, 6, 7, 8) show that while the perturbed images look identical to clean ones, the model outputs become semantically nonsensical (e.g., identifying a "tile wall" as "glass," or a "girl with a towel" as "a man fighting").
  • Sentence-BERT Analysis: Using SBERT to measure semantic similarity (which is more robust to lexical variations than CIDEr-D), the NUM perturbation consistently yielded the lowest similarity scores, confirming a deep semantic breakdown rather than just surface-level noise.
• Precision Independence: Experiments transferring perturbations across bfloat16, float16, and float32 models showed that while float32 performed slightly better, the degradation persisted across all precisions, confirming that the instability is not solely a function of low-bit representation but also of the model's internal functional dynamics.

5. Significance and Implications

• Hidden Costs of Efficiency: The paper highlights a critical trade-off in deploying LVLMs. The industry standard of using half-precision (float16) for efficiency creates a hidden vulnerability that can be exploited to cause catastrophic failure.
• Beyond Adversarial Perturbations: This work challenges the current paradigm of robustness evaluation. Standard adversarial defenses (like adversarial training against FGSM/PGD) may not protect against numerical instability, as the attack mechanism is fundamentally different (targeting arithmetic accumulation rather than decision boundaries).
• Future Directions: The authors suggest that mitigating this issue requires new approaches, potentially involving Lipschitz-constrained architectures (though noted as computationally difficult for deep models) or novel detection mechanisms for numerical stress. It calls for a re-evaluation of model reliability in safety-critical applications where "silent" numerical failures could have severe consequences.

In summary, this paper uncovers a fundamental fragility in modern MLLMs where the very optimizations used to make them efficient (low-precision arithmetic) can be weaponized to induce massive performance collapse through carefully crafted, imperceptible input perturbations.