📊 statistics

Why Does Agentic Safety Fail to Generalize Across Tasks?

This paper argues that agentic safety fails to generalize across tasks not merely due to training limitations, but because the inherent complexity of mapping task specifications to safe execution is fundamentally higher than that of execution alone, a conclusion supported by theoretical analysis of Lipschitz constants and empirical experiments with neural network and LLM agents.

Original authors: Yonatan Slutzky, Yotam Alexander, Tomer Slor, Yoav Nagel, Nadav Cohen

Published 2026-05-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yonatan Slutzky, Yotam Alexander, Tomer Slor, Yoav Nagel, Nadav Cohen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to drive a car. You want it to be able to go to any destination you give it, whether it's a sunny park or a rainy highway. This is the "multi-task" setting: the robot learns to be a general driver, not just a driver for one specific route.

The paper asks a very specific question: If we teach the robot to drive safely (avoiding crashes and obstacles), will it stay safe when we send it to a new destination it has never seen before?

The short answer the authors found is: No, not necessarily.

Here is the breakdown of their discovery using simple analogies:

1. The "Safe Teacher" vs. The "Reckless Teacher"

The researchers set up an experiment with two types of teachers:

The Reckless Teacher: Teaches the robot how to get to the destination as fast as possible, ignoring traffic lights or potholes.
The Safe Teacher: Teaches the robot how to get to the destination while strictly avoiding potholes and obeying traffic rules.

They found that when the robot practiced on the specific roads the teachers knew, it learned perfectly from both. It could mimic the reckless teacher and the safe teacher equally well.

The Problem: When they sent the robot to a brand new road (a task it had never seen in training), the robot that learned from the Safe Teacher struggled much more than the one that learned from the Reckless Teacher. The safe robot would often get confused, make mistakes, or fail to navigate the new road safely, even though it was great at the old roads.

2. The "Complex Map" Analogy

Why does this happen? The authors argue it's not because the robot is "dumb" or because the training wasn't good enough. It's because safety is inherently more complex than just "getting the job done."

Getting the job done is like drawing a straight line from Point A to Point B. If you know how to draw a straight line to one house, you can usually figure out how to draw a straight line to a new house nearby. The relationship between the destination and the path is simple.
Doing it safely is like drawing a path that must avoid a specific set of invisible, shifting mines. The relationship between the destination and the "safe path" is much more tangled. A tiny change in the destination might require a completely different, complex detour to avoid the mines.

The paper proves mathematically that the "map" connecting a task to a safe solution is much more "jagged" and sensitive to changes than the map connecting a task to a simple solution. In math terms, the "slope" of the safe map is steeper. This means that if you make a small mistake in understanding the new task, your safe solution can go wildly wrong, whereas your simple solution might just be slightly off.

3. The "Smooth vs. Rough Terrain" Metaphor

Imagine you are learning to walk.

Task Execution is walking on a flat, smooth sidewalk. If you learn to walk on one sidewalk, you can easily walk on a new one.
Safety is walking on a tightrope while juggling. If you learn to juggle on a specific tightrope, you might not be able to juggle on a new tightrope that is slightly different. The rules of "don't fall" make the relationship between "where you are" and "what you do" incredibly fragile.

4. What This Means for AI

The paper concludes that we cannot simply assume that if an AI is safe on the tasks we trained it on, it will be safe on new tasks.

The Myth: "If we train the AI on enough safe examples, it will generalize safety to everything."
The Reality: Safety is a fundamentally harder skill to generalize than performance. Just because an AI can do a new task doesn't mean it can do that new task safely.

The authors suggest that current methods (like just showing the AI more examples of safe behavior) might not be enough. We might need to invent entirely new ways to teach AI how to understand the complex relationship between "what I need to do" and "how to do it without breaking anything."

In summary: Teaching an AI to be safe is like teaching it to walk a tightrope. It might get good at walking on the specific tightrope you built, but when you put it on a new, slightly different tightrope, it's much more likely to fall than if you had just taught it to walk on the ground. The paper proves this isn't a training error; it's a fundamental property of safety itself.

Technical Summary: Why Does Agentic Safety Fail to Generalize Across Tasks?

Problem Statement

As Artificial Intelligence (AI) evolves from passive data processing to agentic settings where systems interact with dynamic environments to perform long-term tasks, a critical challenge has emerged: cross-task generalization of safety. Modern agentic AI is predominantly multi-task, where the specific task is specified only at test time. While agents often generalize their ability to execute tasks effectively across unseen scenarios, empirical evidence suggests that their ability to execute these tasks safely frequently fails to generalize.

Current approaches often rely on imitation learning from a "safe teacher" (e.g., a human or a verified model) to instill safety behaviors. However, this paper investigates whether the failure of safety to generalize is merely a limitation of current training techniques or an inherent property of safety itself. The core question is: Is the relationship between a task specification and its safe execution fundamentally more complex than the relationship between a task and its execution alone?

Methodology

The authors employ a dual approach combining theoretical analysis and empirical validation across three distinct settings.

1. Theoretical Analysis

The paper formalizes the problem using Linear-Quadratic Control with $H_\infty$ -Robustness.

Setup: The authors analyze a Linear Dynamical System (LDS) where an agent must maximize a task-dependent quadratic reward (defined by a state cost matrix $Q$ ) while handling potential disturbances (risk handling).
Comparison: They compare two mappings from task specification ( $Q$ $Q$ ) to an optimal controller:
- $K_{unsafe}(Q)$ : The Linear-Quadratic Regulator (LQR) which ignores safety (disturbances).
- $K_{safe}(Q)$ : The optimal controller ensuring $H_\infty$ -robustness (handling disturbances).
Metric: The difficulty of generalization is analyzed through the Lipschitz constant of these mappings. A higher Lipschitz constant implies that small changes in the task specification lead to large changes in the optimal controller, making generalization inherently more difficult.
Proof Strategy: The authors derive an upper bound for the Lipschitz constant of the unsafe mapping ( $K_{unsafe}$ ) and a lower bound for the safe mapping ( $K_{safe}$ ). They prove that under specific technical conditions (stability margin, symmetric commutativity, and alignment regularity), the Lipschitz constant of the safe mapping is strictly larger than that of the unsafe mapping by a coefficient that can be arbitrarily large.

2. Empirical Validation

The theoretical findings are corroborated through experiments in three settings spanning different agent types and safety notions:

Linear-Quadratic Control: A direct mirror of the theoretical setup, learning to control an LDS with $H_\infty$ -robustness.
Quadcopter Navigation: A neural network agent trained to navigate a simulated quadcopter to task-dependent locations while avoiding "no-go" regions (risk avoidance).
CRM via LLM Agent: An LLM agent (LLaMA-3.2) fine-tuned to perform Customer Relationship Management (CRM) tasks while avoiding high-risk situations (e.g., saving PII without notification) and handling materialized risks (e.g., invalid inputs).

In all settings, agents were trained via imitation learning to match either a "safe teacher" (adhering to constraints) or an "unsafe teacher" (ignoring constraints). Generalization was measured by the imitation error on tasks unseen during training ( $\Theta_{test}$ ).

Key Contributions

Theoretical Contributions

Lipschitz Separation: The paper provides the first theoretical proof that the mapping from task specification to an optimal controller has a higher Lipschitz constant under safety requirements than without them.
Inherent Difficulty: This result establishes that generalization across tasks is fundamentally more difficult with safety requirements, regardless of how well safety is learned on training tasks. The complexity arises from the intrinsic relationship between the task and the safe execution, not just the learning algorithm.
New Bound: As a technical contribution, the authors establish an upper bound for the Lipschitz constant of the standard Linear-Quadratic Regulator (LQR) as a function of its state cost matrix $Q$ .

Empirical Contributions

Cross-Setting Validation: The experiments demonstrate that the separation in Lipschitz constants translates to a significant separation in cross-task generalization errors. Agents imitating safe teachers exhibit significantly higher error rates on unseen tasks compared to those imitating unsafe teachers, even when training performance is comparable.
Robustness of Findings: The phenomenon holds across different agent architectures (neural networks, LLMs) and different safety definitions (risk handling via robustness, risk avoidance via constraints).

Results

Theoretical: The derived lower bound for the Lipschitz constant of $K_{safe}$ consistently exceeds the upper bound for $K_{unsafe}$ . The ratio $Lip(K_{safe}) / Lip(K_{unsafe})$ is consistently greater than 1, often significantly so.
Experimental:
- In the Linear-Quadratic setting, the test error for the safe teacher was orders of magnitude higher than for the unsafe teacher (e.g., ~1265 vs. ~14 in infinite sample limit; ~4018 vs. ~33 in finite sample).
- In Quadcopter Navigation, the safe agent's test error was ~141.74 compared to ~22.90 for the unsafe agent.
- In the CRM LLM setting, the safe agent's test error was ~25.94 compared to ~16.93 for the unsafe agent.
- Crucially, training errors (imitation on seen tasks) were similar for both safe and unsafe teachers, isolating the generalization gap as the primary failure mode.

Significance and Claims

The paper argues that the failure of agentic safety to generalize is not merely a limitation of current training techniques (such as insufficient data or suboptimal architectures) but reflects an inherent property of safety itself.

Complexity of Safety: The relationship between a task and its safe execution is inherently more complex than the relationship between a task and its execution alone. This complexity manifests as a higher sensitivity of the safe controller to variations in task specifications.
Implications for Practice: The authors suggest that current efforts to enhance safety by simply training on a larger subset of tasks may be insufficient. Because safety generalization is a distinct and harder problem than execution generalization, fundamentally different approaches are required.
Future Direction: A promising avenue suggested is learning representations for both task specifications and safety characteristics that simplify the relationship between them, thereby reducing the Lipschitz constant of the mapping.

The paper concludes that cross-task generalization of safety should be treated as a distinct problem, separate from the generalization of execution ability or the generalization of safety within a single task.