Technical Summary: ARB4WM – An Adversarial Robustness Benchmark for World Models in Continuous Control

1. Problem Statement

World models, particularly Dreamer-style agents, are increasingly deployed in safety-critical continuous control systems (e.g., robotics, autonomous driving) due to their ability to learn latent dynamics for planning and decision-making. Unlike model-free agents that map observations directly to actions, world models encode visual inputs into compact latent states, learn recurrent dynamics, and optimize behavior through imagined trajectories.

While prior work has established that reinforcement learning (RL) agents are vulnerable to adversarial observations, existing evaluations are insufficient for world models. Most current benchmarks focus on model-free policies or treat the agent as a simple policy network, evaluating only action-space robustness. This approach fails to capture failure modes specific to the world-model pipeline, where adversarial perturbations can corrupt:

• Recurrent belief states (latent dynamics).
• Value predictions used for imagined rollouts.
• The consistency between posterior and prior distributions in the Recurrent State-Space Model (RSSM).

Consequently, there is a lack of a unified framework to diagnose where and how adversarial visual perturbations disrupt the latent decision-making pipeline across different world-model architectures.

2. Methodology: The ARB4WM Framework

The authors present ARB4WM, a repeatable robustness testing framework designed to evaluate world-model agents under visual perturbations. The framework evaluates four Dreamer-family agents (Dreamer, R2-Dreamer, Dreamer-InfoNCE, and Dreamer-Pro) across 20 tasks from MetaWorld and the DeepMind Control Suite (DMC).

2.1. Unified Attack Interface

ARB4WM defines a unified interface that accesses internal components of the Dreamer pipeline during inference and imagination rollouts. This allows for targeted attacks on specific architectural elements rather than just the final action output.

2.2. Five White-Box Attack Objectives

The framework introduces five distinct loss objectives to target different components of the world-model pipeline:

1. Policy Divergence Attack (PDA): Maximizes the Kullback–Leibler (KL) divergence between the clean policy distribution and the adversarial policy distribution ($\mathcal{L}_{PDA} = D_{KL}(\pi_{clean} \parallel \pi_{adv})$).
2. Policy Entropy Attack (PEA): Maximizes the entropy of the adversarial policy to induce indecisiveness ($\mathcal{L}_{PEA} = \mathcal{H}(\pi_{adv})$).
3. Value Suppression Attack (VSA): Minimizes the critic's predicted state value to destabilize long-horizon planning ($\mathcal{L}_{VSA} = -V(\tilde{f}_t)$).
4. Latent Drift Attack (LDA): Maximizes the squared $L_2$ distance between clean and adversarial latent states (both stochastic $z_t$ and deterministic $h_t$) to corrupt the belief state ($\mathcal{L}_{LDA} = \|z_t - z^{clean}_t\|^2_2 + \|h_t - h^{clean}_t\|^2_2$).
5. Dynamics Consistency Attack (DCA): Maximizes the KL divergence between the RSSM posterior and prior distributions to break the consistency of latent imagination ($\mathcal{L}_{DCA} = D_{KL}(q(z_t) \parallel p(z_t))$).

2.3. Optimization and Temporal Protocols

• Optimizers: The framework employs both single-step (FGSM) and multi-step iterative (PGD) gradient-based attacks.
• Temporal Exposure Modes: To simulate realistic constraints and study recurrent state propagation, the framework evaluates:
  • Full-frame: Perturbing every frame (maximum pressure).
  • Half-sequence: Perturbing only the first or second half of the episode to test recovery and long-term propagation.
  • Sparse-periodic: Perturbing every $n$ frames to test the resilience of latent dynamics against intermittent attacks.

2.4. Evaluation Metrics

The framework reports:

• Clean-normalized Area Under Curve (nAUC): Aggregates performance across perturbation budgets.
• Defense Recovery: Measures the extent to which input defenses (e.g., JPEG compression, median filtering) restore performance.
• Adaptive-Defense Gap: Quantifies the drop in defense effectiveness when the attacker is aware of the defense.
• Policy Saliency Diagnostics: Analyzes how adversarial objectives alter the spatial evidence (saliency maps) used by the policy.

3. Key Results

Experiments across 20 tasks and four agents yielded the following findings:

• Component-Level Vulnerabilities: Attacks targeting value estimation (VSA), latent representations (LDA), and RSSM dynamics (DCA) were found to be as damaging as direct policy disruption (PDA/PEA). In many cases, VSA and PEA were the strongest attack objectives, indicating that corrupting the critic or policy uncertainty is highly effective.
• Temporal Propagation: Full-frame attacks caused the most significant degradation. Crucially, early perturbations (first-half attacks) were often more damaging than late perturbations, suggesting that errors injected into the recurrent latent state persist and propagate even after the attack stops. Sparse attacks (every 2 frames) also caused substantial degradation, while higher intervals (every 4-6 frames) allowed the latent state to re-anchor.
• Architecture Sensitivity: R2-Dreamer demonstrated the strongest average robustness across benchmarks, while Dreamer-Pro was consistently the weakest, despite its design for improved representation learning. This suggests that nominal representation quality does not guarantee adversarial stability.
• Limitations of Defenses: Input-level defenses (JPEG, median filtering) provided partial recovery under non-adaptive attacks but offered limited protection against adaptive attacks. The effectiveness of defenses was highly task-dependent; for example, they helped in manipulation tasks with redundant visual cues but failed in balance-control tasks requiring precise feature extraction.
• Saliency Shifts: Adversarial objectives altered the spatial sensitivity of the policy. Attacks on value and dynamics consistency (VSA, DCA) tended to shift the policy's attention away from the target region more significantly than policy-level attacks.

4. Significance and Claims

The paper claims that ARB4WM addresses a critical gap in the safety assessment of world-model agents by moving beyond simple action-space robustness. Its primary contributions and implications are:

1. Holistic Safety Assessment: The authors argue that robustness evaluation for world models must cover the full latent decision-making pipeline, including policy outputs, critic estimates, latent representations, and dynamics consistency. Relying solely on action-space metrics misses internal failure modes that propagate through the system.
2. Temporal Risk Awareness: The framework highlights that temporal exposure protocols are essential for risk assessment. The persistence of latent state corruption means that safety evaluations must consider whether a controller can recover from intermittent or early-stage disturbances.
3. Defense Realism: The results suggest that input-level defenses should be treated as partial mitigations rather than certification mechanisms. The significant "adaptive-defense gap" indicates that defenses must be evaluated against attackers who can optimize through the defense pipeline.
4. Architectural Implications: The findings imply that the choice of representation learning objectives (e.g., redundancy reduction in R2-Dreamer vs. prototype learning in Dreamer-Pro) directly influences the local geometry of the encoder-RSSM-head composition, thereby affecting adversarial robustness.

In summary, ARB4WM provides a standardized, repeatable workflow to diagnose specific failure modes in world-model agents, offering a foundation for developing more robust visual model-based control systems for industrial and safety-critical applications.