Technical Summary: Security of World-Model-Based Embodied AI

Problem Statement

Embodied Artificial Intelligence (AI) is transitioning from reactive perception to predictive cognition, utilizing world models to compress observations, simulate action-conditioned futures, and plan beyond immediate control. While this predictive layer enhances capability, it introduces a critical new security boundary. Unlike conventional digital systems where corruption leads to classification errors or text generation failures, compromised world models in embodied AI can cause unsafe physical motion, collisions, and harmful human-robot interactions.

The core problem addressed is that existing security literature treats world-model attacks (e.g., adversarial perception, data poisoning, video generation attacks) as isolated phenomena across disparate fields (robotics, autonomous driving, generative AI). Furthermore, world models are often viewed strictly as either attack targets or safety tools, ignoring their dual nature: they can be the source of poisoning (via synthetic data generation) and the mechanism for safety verification (via runtime prediction), yet both roles are vulnerable. There is a lack of a unified framework to trace how threats propagate from data construction through to physical execution, and how a corrupted internal model can create "predictive safety illusions" where unsafe actions appear certified.

Methodology

This paper proposes a lifecycle taxonomy for world-model-based embodied AI security. Rather than focusing on specific architectures (e.g., RSSM, Diffusion, Transformers), the authors define a world-model-based system as any agent that maintains a predictive structure to simulate, score, or execute embodied behavior.

The methodology involves:

1. Defining Security Objects: Establishing seven core properties that must be protected: State Integrity, Dynamics Fidelity, Affordance Correctness, Constraint Compliance, Trajectory-Ranking Integrity, Uncertainty Calibration, and Feedback Provenance.
2. Lifecycle Decomposition: Mapping security threats across six distinct stages of the agent's operation:
   • Data Construction and Pretraining
   • Model Training and Representation
   • State Grounding (Runtime perception)
   • Dynamic Prediction and Imagination
   • Trajectory Evaluation and Action Selection
   • Execution, Feedback, and Agentic Extension (Memory/Tools)
3. Threat Mapping: Systematically categorizing established attack families (poisoning, backdoors, adversarial examples, prompt injection, sensor spoofing, supply-chain attacks) and analyzing how they manifest specifically within the context of world models (e.g., how a backdoor triggers a false future rollout rather than just a misclassification).
4. Dual-Use Analysis: Analyzing world models simultaneously as attack targets and as safety mechanisms (e.g., runtime safety checkers), highlighting the specific risks when a safety checker itself is compromised.

Key Contributions

The paper makes four primary contributions to the field:

1. A Unified Lifecycle Framework: It organizes scattered literature into a single decision-making lifecycle, covering data construction, training, grounding, imagination, evaluation, execution, and long-term adaptation.
2. Mapping Attacks to Security Objects: It connects established attack families from adversarial ML, VLA security, and robotics to specific world-model security objects (e.g., mapping "trajectory-ranking attacks" to "Trajectory-Ranking Integrity").
3. Dual-Use Analysis of World Models: It explicitly analyzes world models as both attack targets (where the model is corrupted) and poisoning amplifiers (where generative world models create synthetic training data that spreads attacks) and safety mechanisms (where compromised safety checkers generate false certificates of safety).
4. Evaluation and Defense Synthesis: It outlines specific evaluation protocols for measuring safety failures (e.g., "predicted-safe-but-actually-unsafe" rates) and structures defenses across provenance, robust grounding, uncertainty-aware prediction, and trajectory gating.

Five Recurring Insights

The survey structures its analysis around five key insights that define the unique security challenges of world-model-based systems:

• SP (Semantic-to-Simulation-to-Action Gap): Semantically plausible plans or generated futures may be physically unsafe; textual refusal or visual quality is insufficient without dynamic feasibility.
• SD (State- and Uncertainty-Conditional Risk): Attack success is highly dependent on the current state (pose, lighting, layout) and model uncertainty; a trigger benign in one state may be hazardous in another.
• PA (Rollout-to-Execution Amplification): Small corruptions in state estimation or latent dynamics can compound across imagined rollouts and real closed-loop execution.
• NC (Trajectory-Level Non-Compositionality): Locally safe steps do not guarantee a safe long-horizon trajectory; attacks can target the ranking of multi-step futures rather than single-step predictions.
• PI (Predictive Safety Illusion): When a world model acts as a safety checker, a confident but incorrect prediction can certify an unsafe action, creating a false sense of security.

Results and Analysis

The paper does not present new experimental results but synthesizes findings from existing literature to demonstrate the scope of the threat landscape:

• Attack Propagation: Attacks can enter at any stage (e.g., poisoned training data) and manifest later as corrupted state grounding, false future rollouts, or unsafe action selection.
• Backdoor Persistence: Backdoors in world models are particularly dangerous because they can remain dormant during training and validation, activating only under specific state conditions to corrupt future predictions or trajectory rankings.
• Safety Checker Subversion: The paper highlights that safety filters relying on learned dynamics are vulnerable to "dynamics-channel subversion," where an attacker manipulates the transition model to make unsafe actions appear safe in simulation.
• Benchmark Gaps: Current benchmarks lack standardized protocols for evaluating "corrupted imagined futures" and "predicted-safe-but-actually-unsafe" rates, making cross-method comparison difficult.

Significance and Claims

The paper claims that the security of embodied AI cannot be addressed by securing individual components (sensors, policies, or planners) in isolation. Instead, the predictive core (the world model) must be treated as the central security boundary.

The authors argue that:

1. World models are dual-use: They are essential for safety (predicting risks) but also introduce new attack surfaces (generating false safety certificates or poisoning synthetic data).
2. Defense requires a lifecycle approach: Effective defense must span from data provenance and training hygiene to runtime uncertainty calibration and feedback auditing.
3. New metrics are required: Traditional metrics like task success or attack success rate are insufficient. Security evaluation must measure the rate of "predicted-safe-but-actually-unsafe" executions and the robustness of safety certificates against adaptive attacks.

The paper concludes that building secure world-model-based embodied AI requires a shift from capability-centric surveys to security-centric frameworks that account for the propagation of digital compromise into physical risk.