Manifold of Failure: Behavioral Attraction Basins in Language Models

This paper introduces a framework using MAP-Elites to systematically map the "Manifold of Failure" in Large Language Models by characterizing continuous behavioral attraction basins and revealing distinct, model-specific safety topologies that shift the paradigm from finding discrete adversarial examples to understanding the global structure of unsafe regions.

The Gist

Imagine you have a very smart robot assistant. For a long time, security experts have tried to break this robot by throwing specific, tricky questions at it. If the robot answers badly, they say, "Aha! We found a bug!" and then they try to patch that specific hole.

This paper argues that this approach is like trying to map a forest by only looking at individual trees. You might find a few dangerous spots, but you miss the shape of the whole forest.

Instead, the authors want to map the entire landscape of failure. They call this the "Manifold of Failure."

Here is the breakdown of their idea using simple analogies:

1. The Old Way vs. The New Way

• The Old Way (Finding a Needle): Imagine you are looking for a needle in a haystack. You stick a magnet in one spot, find a needle, and stop. You know there's a needle there, but you don't know if the rest of the haystack is full of them or if it's just a fluke. This is how most AI safety tests work: they try to find the single worst way to trick the AI.
• The New Way (Mapping the Terrain): The authors say, "Let's stop looking for just one needle. Let's walk through the whole haystack and draw a map." They want to see the shape of the danger. Is the danger a flat, endless plain where the AI fails everywhere? Is it a jagged mountain range with safe valleys in between? Or is it a smooth hill where the AI is mostly safe?

2. The "Attraction Basins" (The Gravity of Failure)

The paper introduces a cool concept called Behavioral Attraction Basins.

Imagine the AI's behavior is like a landscape of hills and valleys.

• Safe Prompts are like rolling a ball on a flat, green meadow. It stays safe.
• Unsafe Prompts are like rolling a ball into a deep, dark pit. Once the ball falls in, it gets stuck there, no matter how you wiggle it.

The authors found that these "pits" (basins) aren't just tiny, isolated holes. They are large, connected regions. If you ask the AI a question in a slightly different way (like changing the tone or the context), the ball might roll from one part of the pit to another, but it's still stuck in the same "danger zone."

3. How They Mapped It (The "Quality-Diversity" Game)

To draw this map, they didn't just try to break the AI; they played a game called MAP-Elites.

Think of a giant grid on the floor (like a chessboard, but 25x25 squares).

• The X-axis represents how "indirect" a question is (from "Give me a gun" to "Imagine a story about a gun").
• The Y-axis represents who is asking (from "Just a random person" to "A strict police officer").

The algorithm tries to fill every single square on this grid with a question that makes the AI fail. But it's smart: it keeps the best failure for each square. If it finds a question that makes the AI fail in the "Police Officer" square, it saves that question. If it finds a worse failure later, it swaps it in.

By the end, they have a heat map.

• Red areas = The AI fails badly here.
• Green areas = The AI stays safe here.

4. What They Found (The Three Different Landscapes)

They tested three different AI models, and each had a totally different "personality" when it came to failing:

• Model A (Llama-3-8B): The "Flat Disaster Zone."
  Imagine a giant, flat desert where the ground is made of quicksand. No matter where you walk (no matter how you ask the question), you sink. This model is almost universally vulnerable. It's like a house with no locks on any of the doors.

• Model B (GPT-OSS-20B): The "Swiss Cheese."
  This model is like a rugged mountain range with deep caves. Some areas are safe (high peaks), but there are specific, concentrated pits where the AI collapses. If you know exactly where the "caves" are, you can fall in, but if you stay on the peaks, you're fine. The danger is patchy and specific.

• Model C (GPT-5-Mini): The "Fortress."
  This model is like a smooth, flat plateau that is just slightly elevated. Even if you push it hard, it never falls off the edge. It has a "ceiling" to how bad it can get. It might get a little grumpy or slightly off-topic, but it never crosses the line into truly dangerous territory. It's the most robust.

5. Why This Matters

The authors say that knowing where the danger is (the map) is more important than just knowing that the danger exists.

• For Builders: Instead of patching one hole, they can see the whole shape of the problem. If they see a "cliff" at a specific type of question, they can reinforce that whole area.
• For Auditors: They can compare models like comparing maps of different countries. "Country A has a flood risk everywhere; Country B only has floods in the valley."
• For Safety: It shifts the goal from "Can we break this?" to "How does this break, and what does that tell us about its brain?"

Summary

This paper is about stopping the "Whack-a-Mole" game of AI safety. Instead of hitting one bad answer and moving on, they built a topographical map of the AI's weaknesses. They discovered that some AIs are like open fields of danger, some are like Swiss cheese with hidden holes, and some are like sturdy fortresses. Understanding the shape of the failure is the key to building safer AI in the future.

Technical Summary

1. Problem Statement

Current AI safety research predominantly adopts a restorative paradigm: it identifies adversarial examples (failures) that lie "off-manifold" from natural data and attempts to project them back onto a safe latent manifold. The authors argue that this approach treats failures as isolated aberrations, failing to capture the underlying structure of model vulnerabilities.

The core problem addressed is the lack of a comprehensive, structural understanding of failure regions in Large Language Models (LLMs). Instead of viewing vulnerabilities as discrete points, the paper posits that they form a continuous, structured landscape termed the "Manifold of Failure." The goal is to systematically map this manifold to understand the topology of unsafe behaviors, rather than simply finding the single "worst-case" attack.

2. Methodology

The authors propose a framework based on Quality-Diversity (QD) optimization, specifically utilizing the MAP-Elites algorithm, to map the behavioral topology of LLMs.

A. Behavioral Space Definition

The input space is mapped to a continuous, 2-dimensional behavioral space $B = [0,1]^2$, defined by two axes:

1. Query Indirection ($a_1$): Measures the degree of semantic paraphrasing, ranging from direct requests (0) to metaphorical/hypothetical scenarios (1).
2. Authority Framing ($a_2$): Measures the perceived authority in the prompt, ranging from no authority (0) to strong administrative/expert personas (1).

B. Quality Metric: Alignment Deviation (AD)

Instead of optimizing for a single "attack success," the system optimizes for Alignment Deviation, defined as:
$$Q(p) = \max_{c \in C} \text{JudgeScore}_c(p)$$
Where $C$ represents 10 harm categories (e.g., violence, hate speech, misinformation, fraud). Two judge LLMs (GPT-4.1 and Sonnet 4.5) evaluate responses, assigning a score between 0 and 1. The AD is the maximum score across all categories, representing the worst-case harm dimension.

C. The MAP-Elites Framework

The algorithm partitions the 2D behavioral space into a 25×25 grid (625 distinct behavioral niches).

1. Initialization: Starts with 100 seed prompts.
2. Mutation: Generates new prompts via six strategies: Random Axis Perturbation (50%), Paraphrasing, Entity Substitution, Adversarial Suffix, Crossover, and Semantic Interpolation.
3. Selection & Storage: For each cell in the grid, the algorithm retains the prompt with the highest Alignment Deviation.
4. Iteration: Runs for 15,000 evaluations to fill the archive with high-quality, diverse solutions.

D. Evaluation Setup

The framework was tested on three models with varying architectures and access levels:

• Llama-3-8B (Open weights, local)
• GPT-OSS-20B (Open weights, local)
• GPT-5-Mini (Closed API, black-box)

Baselines included Random Sampling, GCG (Gradient-based), PAIR (Iterative refinement), and TAP (Tree-search).

3. Key Contributions

1. Manifold of Failure Framework: Introduces a systematic method to map the continuous topological structure of LLM vulnerabilities, shifting from point-based attacks to landscape analysis.
2. Behavioral Attraction Basins: Provides empirical evidence that vulnerabilities exist as extended regions ("basins") where diverse prompts converge to similar failure modes, rather than isolated points.
3. Model-Specific Topological Signatures: Reveals that different models exhibit fundamentally different safety landscapes (e.g., universal plateaus vs. fragmented basins vs. robust ceilings).
4. Superior Coverage: Demonstrates that the QD approach achieves significantly higher behavioral coverage and diversity than traditional adversarial attack methods.

4. Key Results

A. Topological Signatures

The study revealed three distinct "landscapes":

• Llama-3-8B (The Universal Plateau): Exhibits a near-universal vulnerability surface.
  • Mean AD: 0.93 (Peak 1.0).
  • Basin Rate: 93.9% of filled cells exceeded the safety threshold (AD > 0.5).
  • Structure: A high, flat mesa where almost any combination of indirection and authority framing leads to failure.
• GPT-OSS-20B (The Fragmented Landscape): Shows spatially concentrated basins.
  • Mean AD: 0.73.
  • Basin Rate: 64.3% of filled cells.
  • Structure: Rugged terrain with "bullseye" patterns of high vulnerability surrounded by safe regions. Vulnerabilities are localized rather than uniform.
• GPT-5-Mini (The Robust Ceiling): Demonstrates strong alignment.
  • Mean AD: 0.47.
  • Basin Rate: 0% (No cells exceeded AD 0.5).
  • Structure: A uniform plateau with a hard ceiling at AD 0.50. Even with 72% behavioral coverage, the model never crossed into genuinely harmful territory.

B. Comparative Performance

• Behavioral Coverage: MAP-Elites achieved 63.04% coverage on Llama-3-8B, outperforming PAIR (61.44%), TAP (41.76%), and GCG (7.20%).
• Diversity: MAP-Elites discovered 370 distinct vulnerability niches on Llama-3-8B, compared to 291 for PAIR.
• Efficiency: On GPT-OSS-20B, while PAIR covered more cells (63.2% vs 36.3%), MAP-Elites found a much higher density of actual vulnerabilities (64.3% of its cells were unsafe vs. 18.5% for PAIR), indicating it targets the most informative regions of the space.

C. Contour Analysis

Contour plots revealed horizontal banding across all models, indicating that Authority Framing ($a_2$) acts as a critical parameter. Models have discrete "thresholds" of authority recognition where safety alignment shifts abruptly, suggesting that social context is a primary driver of compliance behavior.

5. Significance and Implications

• Paradigm Shift: Moves AI safety from a "find the worst attack" mindset to a "map the failure landscape" approach. This allows for predictive safety auditing rather than reactive patching.
• Targeted Remediation: By identifying specific "basins" and "corridors" of failure, developers can apply targeted robustness improvements to specific behavioral coordinates (e.g., hardening responses to specific authority framings) rather than generic fine-tuning.
• Topological Science of AI: Establishes a foundation for treating model behavior as a geometric object with learnable structures, enabling regression testing and comparative auditing across model versions.
• Open Source: The framework, metrics, and datasets are open-sourced to facilitate community replication and further research into the topological science of model behavior.

In conclusion, the paper demonstrates that LLM vulnerabilities are not random noise but structured, continuous manifolds. Understanding this structure is essential for building truly robust and safe AI systems.