Technical Summary: Test-time Adversarial Takeover (TAKO)

Problem Statement

Diffusion-based action generation has become a foundational component of embodied AI, particularly in visuomotor policies that rely on visual conditioning (e.g., via ResNets or ViTs) to generate actions. While existing adversarial research on robot policies has focused on disruption—perturbing observations to cause task failure or erratic behavior—this paper identifies a more severe threat model: Test-time Adversarial Takeover (TAKO).

In the TAKO scenario, an attacker does not merely disrupt a policy but hijacks it, obtaining a real-time steering interface to turn a frozen robot policy into a remotely piloted instrument. The attacker aims to compose arbitrary, attacker-defined trajectories by switching among a small vocabulary of reusable universal adversarial patches injected into the camera stream. This shifts the security concern from accidental failure to unauthorized, sustained control.

The paper argues that a natural baseline for this threat, Target-Policy Matching (TPM), is fundamentally flawed. TPM attempts to optimize a perturbation to match the actions of a target-shifted policy. However, this fails because:

1. Out-of-Distribution (OOD) Pathology: The victim policy, trained on specific goals, cannot reliably generate the desired behavior when the goal is shifted to an OOD configuration.
2. Accumulative Drift: In a receding-horizon deployment, a static reference policy optimized at training time cannot correct for drift during long-horizon execution, leading to divergence from the target.

Methodology

The proposed TAKO framework operates in two stages: Offline Optimization and Online Compositional Deployment.

1. Offline Optimization: Universal Patch Vocabulary

Instead of matching a specific target trajectory, the attacker learns a "vocabulary" of universal adversarial patches. Each patch corresponds to a directional action primitive (e.g., "move forward," "move left").

• Objective: The patches are optimized to induce a consistent directional bias in the policy's action output, rather than matching a specific target state.
• Loss Function: A directional loss is defined to maximize the mean shift of the action sequence along a specific axis (e.g., $x$ or $y$).
  $$\mathcal{L}_c(\delta_c) = -\sigma_c \cdot \mathbb{E}_{o \sim \mathcal{D}} \left[ \frac{1}{H_a} \sum_{t'=1}^{H_a} \hat{a}_{t', d_c}(o, \delta_c) \right]$$
  Where $\sigma_c$ is the sign of the desired direction, $d_c$ is the action axis, and $\hat{a}$ is the action sequence generated by the patched observation.
• Differentiable Inference: Gradients are backpropagated through the frozen diffusion inference chain (including the vision encoder and the denoiser) to the patch parameters. This works across different inference families (DDPM, DDIM, and Flow Matching) because the perturbation acts on the visual conditioning pathway, biasing the iterative denoising process.

2. Online Compositional Deployment

At test time, the attacker holds the optimized vocabulary $\{\delta^*_c\}$.

• Control Interface: A human operator (or automated agent) selects a command $u_t$ from the vocabulary (or a "no patch" state) at each replanning step based on the current state.
• Closed-Loop Correction: Unlike open-loop attacks, the operator can switch patches in real-time to correct for drift. The victim policy treats the patched image as a clean input, generating smooth action sequences that follow the operator's chosen trajectory.
• Mechanism: The attack exploits the fact that the adversarial bias introduced via visual conditioning persists through the iterative generative inference, steering the policy away from its trained behavior toward the attacker's desired direction.

Key Contributions

1. New Adversarial Paradigm: The paper introduces TAKO, shifting the objective from causing policy failure to obtaining sustained, operator-controllable authority over policy behavior.
2. Steering Interface: It demonstrates that diffusion-based visuomotor policies admit a simple steering interface: a pre-optimized vocabulary of adversarial patches that can be switched online to compose arbitrary trajectories.
3. Generalizability: The takeover paradigm is validated as not task or architecture-specific. It works across:
   • Tasks: 2D manipulation (PushT), simulated aerial delivery (DeliverDrone), simulated ground navigation (SimNav), and physical-world ground navigation (RealNav).
   • Encoders: ResNet-18 and EfficientNet-B0 + Transformer.
   • Inference Families: DDPM, DDIM, and Flow Matching.

Experimental Results

The authors evaluated TAKO against the TPM baseline and the original clean policies across four tasks.

• Baseline Failure: The TPM baseline achieved 0% success (0/40 trials) across all tasks. The victim policies failed to generalize to the shifted target configurations, and the static reference trajectories accumulated drift.
• TAKO Success: Human operators using the TAKO interface achieved 100% success (40/40 trials) in steering the robots to attacker-defined targets that were never part of the training distribution.
• Directional Bias: The optimized patches induced consistent directional biases with success rates ranging from ~65% to 100% depending on the task and inference method, confirming the patches function as a reusable action vocabulary.
• Physical Realizability: The attack was successfully demonstrated on a physical iRobot Create 3 platform (RealNav), proving the vulnerability extends beyond simulation.

Significance and Claims

The paper claims that TAKO represents a critical shift from a safety problem (preventing accidental failure) to a security problem (preventing unauthorized takeover).

• Structural Vulnerability: The findings suggest that adversarial bias introduced through visual conditioning can persist sufficiently during generative inference to support sustained steering. This vulnerability is inherent to the architecture of diffusion policies that rely on visual conditioning, regardless of the specific denoising algorithm.
• Limitations: The authors note that their experiments assume white-box access for offline optimization and digital injection of patches. They acknowledge that realizing this takeover via physical patches or black-box optimization remains a direction for future work.
• Implications: The work motivates the need for stronger evaluation methods, defenses (such as input-space anomaly detection or certified robustness on visual encoders), and security mechanisms for generative robot policies under interactive adversarial control. The authors emphasize that the necessary components for such attacks (universal adversarial patches, differentiable diffusion inference) are already present in public literature, making this a plausible threat for motivated adversaries.