💬 NLP

REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinations

The paper introduces REALISTA, a novel adversarial attack framework that generates realistic hallucination-inducing prompts by optimizing continuous combinations of semantically equivalent editing directions in latent space, thereby overcoming the limitations of existing discrete and continuous methods to effectively target both open-source and large reasoning models.

Original authors: Buyun Liang, Jinqi Luo, Liangzu Peng, Kwan Ho Ryan Chan, Darshan Thaker, Kaleab A. Kinfu, Fengrui Tian, Hamed Hassani, René Vidal

Published 2026-05-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Buyun Liang, Jinqi Luo, Liangzu Peng, Kwan Ho Ryan Chan, Darshan Thaker, Kaleab A. Kinfu, Fengrui Tian, Hamed Hassani, René Vidal

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Magic Trick" of AI Confusion

Imagine you have a very smart, well-trained parrot (the Large Language Model, or LLM). You ask it a simple question: "What is 2 plus 2?" It confidently says, "4."

Now, imagine you ask the exact same question, but you rephrase it slightly: "If you combine two apples with two more apples, how many do you have?" A normal human would still say "4." But this smart parrot suddenly gets confused and says, "5."

This is called a hallucination. The paper asks: Why does the parrot get confused just because we changed the words, even though the meaning is exactly the same?

To find out, researchers need to trick the parrot. But they can't just shout gibberish or tell a fake story, because that changes the game. They need to find a "magic spell" (an adversarial prompt) that sounds perfectly natural and means the exact same thing as the original question, but somehow breaks the parrot's brain.

The Problem: Two Bad Ways to Trick the Parrot

Before this paper, researchers tried two main ways to trick the AI, and both had flaws:

The "Discrete" Approach (The Dictionary Swap):
Imagine trying to rewrite the question by swapping words from a thesaurus. You change "combine" to "merge," "apples" to "oranges."
- The Good: It sounds human and keeps the meaning.
- The Bad: It's like trying to paint a masterpiece using only 10 specific colors. You are limited to the words you have in your dictionary. You might miss the perfect combination of words that would break the AI.
The "Continuous" Approach (The Digital Sludge):
Imagine trying to tweak the question by making tiny, invisible digital adjustments to the AI's internal "thoughts" (its latent space).
- The Good: You have infinite freedom to tweak the thoughts.
- The Bad: When you turn those tweaked thoughts back into words, they often come out as nonsense or gibberish (like "S!mpl&fy (2 + 5)2 −4@2"). The AI understands the gibberish, but it's not a realistic test because real humans don't talk like that.

The Solution: REALISTA (The "Lego" Approach)

The authors created a new method called REALISTA. Think of it as building with Lego bricks inside the AI's brain.

The Lego Bricks (Edit Directions):
Instead of guessing random words or random digital noise, REALISTA first builds a special set of "Lego bricks." Each brick represents a specific, valid way to rephrase a sentence without changing its meaning.
- Example: One brick might be "make it sound more formal." Another might be "ask it as a question instead of a statement." Another might be "add a little bit of drama."
- Crucially, these bricks are input-dependent. If you are asking about math, the bricks are math-rephrasing tools. If you are asking about history, they are history-rephrasing tools.
Mixing the Bricks (Continuous Optimization):
Now, instead of picking just one brick, REALISTA uses math to figure out the perfect mix of these bricks. It asks: "If I use 10% of the 'formal' brick, 5% of the 'drama' brick, and 2% of the 'question' brick, will the AI get confused?"
- This allows for a smooth, infinite search (like mixing paint colors) rather than a limited, choppy search (like picking from a menu).
The Safety Net (The Simplex Constraint):
To make sure the result doesn't turn into gibberish, REALISTA puts the mix on a "safety leash." It ensures the total amount of change is small and that the bricks are only added (never subtracted in a way that breaks logic). This guarantees that the final sentence still sounds like a human wrote it and still means the same thing as the original.

How It Works in Practice

Start: You give the system a normal question (e.g., "What is the capital of France?").
Translate: The system translates this question into the AI's internal "thought language" (latent space).
Build: It looks at its custom Lego set (the edit dictionary) for that specific question.
Optimize: It mathematically mixes the Lego bricks to find the combination that makes the AI answer "The capital of France is London" (a hallucination) while keeping the question sounding natural.
Decode: It translates the mixed "thoughts" back into English.
Result: You get a question like: "Could you tell me, in a straightforward manner, which city serves as the primary seat of government for France?"
- It sounds natural.
- It means the same thing.
- But the AI, for some reason, thinks the answer is "London."

Why This Matters (According to the Paper)

The paper shows that REALISTA is better than previous methods at two things:

Success Rate: It tricks the AI more often than the old "dictionary swap" methods.
Realism: It doesn't produce gibberish like the old "digital sludge" methods.

Most importantly, the paper claims this method works even on advanced "Reasoning" models (the smartest, most complex AIs) that usually ignore simple tricks. It can find the specific, subtle rephrasing that causes even the smartest AI to hallucinate, proving that these models are still fragile when faced with realistic, human-like variations of a question.

In short: REALISTA is a tool that finds the perfect, natural-sounding way to rephrase a question to confuse an AI, by mixing "rephrasing ingredients" in a mathematically precise way, ensuring the result is both effective and realistic.

Technical Summary: REALISTA

Problem Statement

Large Language Models (LLMs) exhibit hallucinations even when responding to benign user queries. Understanding the mechanisms that trigger these failures requires realistic adversarial attacks that satisfy two critical constraints:

Semantic Equivalence: The adversarial prompt must preserve the original semantic intent of the benign prompt.
Semantic Coherence: The adversarial prompt must be fluent and human-like.

Existing methods fail to simultaneously satisfy these constraints. Discrete prompt-based attacks (e.g., SECA) preserve semantic realism but are limited to a sparse, discrete search space, restricting attack diversity. Continuous latent-space attacks explore a richer space but often decode into prompts that are gibberish or semantically distinct from the original, violating the equivalence constraint. The core challenge is to enable broad exploration in the latent space while strictly enforcing semantic realism.

Methodology: REALISTA

The authors propose REALISTA (Realistic Latent Adversarial Attacks), a framework that formulates hallucination elicitation as a constrained optimization problem in the LLM latent space. The method bridges the gap between discrete and continuous approaches through the following components:

1. Input-Dependent Edit Dictionary

Instead of optimizing arbitrary latent vectors, REALISTA constructs a dictionary of interpretable editing directions specific to each input prompt $x_0$ .

Concept Selection: Using a concept optimization procedure (leveraging WordNet and an LLM-based evaluator), the method selects a diverse set of concept keywords relevant to the input.
Latent Directions: For each selected concept, a semantically equivalent rephrasing $x^{(i)}_{SE}$ is generated. The encoder $\phi$ maps these to latent representations, and the editing direction is defined as $z^{(i)} = \phi(x^{(i)}_{SE}) - \phi(x_0)$ .
Dictionary: The set of these directions forms the dictionary $D$ , which spans a subspace of semantically preserving transformations.

2. Continuous Optimization over a Simplex

The attack is formulated as finding an optimal editing strength vector $\delta$ such that the adversarial latent representation $z$ is:
$z = z_0 + D\delta$
where $z_0 = \phi(x_0)$ . The optimization problem is:
$\min_{\delta} \mathcal{L}_{hall}(f_T(x), y^*) \quad \text{s.t.} \quad \delta \in \Delta_\varepsilon$

Objective: Minimize the hallucination loss (e.g., negative log probability of an incorrect answer or a hallucination score from a judge).
Constraint: $\delta$ is constrained to a scaled simplex ( $\delta \succeq 0, \|\delta\|_1 \le \varepsilon$ ). This enforces sparsity and non-negativity, ensuring the perturbation is a mild, convex combination of semantic-preserving edits rather than a large, arbitrary shift.

3. Decoder and Semantic Safeguard

Prompt Reconstruction: An approximate inverse operator (decoder) $\psi$ , implemented using the same LLM as the encoder, maps the perturbed latent $z$ back to the prompt space $x = \psi(z)$ .
Optimization Algorithm: The authors employ Projected Langevin Dynamics (PLD) to solve the non-convex optimization. This involves:
- Gumbel-Softmax Reparameterization: To enable gradient flow through the discrete decoding process.
- Stochastic Exploration: Injecting Gaussian noise to escape local minima in the piece-wise flat optimization landscape.
- Semantic Equivalence Safeguard: During optimization, if a candidate prompt $x$ fails a semantic equivalence check (performed by an instructed LLM), the gradient signal is discarded, preventing the optimizer from drifting into non-equivalent regions.

Key Contributions

Input-Dependent Edit Dictionary: A method to construct a compact, diverse, and valid set of latent editing directions conditioned on the original prompt, enabling continuous modification while preserving semantics.
Simplex-Constrained Optimization: Formulating realistic hallucination elicitation as a continuous optimization problem over a scaled simplex, balancing exploration flexibility with semantic constraints.
Realistic Attack Framework: The integration of the edit dictionary, simplex constraint, and semantic safeguard into the REALISTA algorithm, which successfully navigates the trade-off between attack effectiveness and semantic realism.

Experimental Results

The paper evaluates REALISTA on open-source LLMs (Llama-3, Qwen-2.5) and commercial reasoning models (GPT-5 variants) across 16 MMLU subjects.

Attack Success Rate (ASR): REALISTA achieves superior or comparable ASR compared to state-of-the-art (SOTA) realistic attacks (e.g., SECA). For instance, on Llama-3-8B, REALISTA achieves an ASR@30 of 93.60%, outperforming SECA (82.97%) and LARGO (57.92%).
Semantic Errors: Crucially, REALISTA maintains low Semantic Equivalence Error (SEE) and Semantic Coherence Error (SCE), comparable to SECA and significantly better than continuous baselines like LARGO and ICD, which often exhibit SEE near 100%.
Reasoning Models: REALISTA successfully elicits hallucinations from commercial reasoning models (GPT-5-Nano, GPT-5-Mini) in free-form response settings, a scenario where prior realistic attacks (SECA, LARGO) fail due to their reliance on fixed output formats or access to target model logits.
Concept Analysis: Analysis of activated concepts reveals that successful attacks primarily exploit structural and framing changes (e.g., polarity flipping, logical restructuring, instructional framing) rather than altering factual content.

Significance and Claims

The paper claims that REALISTA demonstrates that realistic hallucination attacks are feasible even in black-box, reasoning-centric deployment scenarios. By bridging the gap between discrete and continuous attack paradigms, the framework provides a more systematic tool for red-teaming LLMs.

The authors emphasize that their work helps characterize failure modes where lexically different but semantically equivalent prompts lead to hallucinations. They note that while the method improves the effectiveness of hallucination elicitation (potentially lowering barriers for misuse), its primary utility lies in developing more robust defense strategies by revealing how models fail under realistic perturbations. The work highlights the importance of latent-space attacks for evaluating LLM reliability in realistic settings where discrete search is insufficient.