Security of Foundation-Model-Powered Embodied Agents: Attack Surfaces, Attacks, Defenses, and Evaluation
This paper presents a trust-boundary-centric survey of foundation-model-powered embodied agents that categorizes security risks into five layers and twelve attack surfaces, analyzes 58 attacks and 61 defenses to reveal research gaps in areas like memory and multi-agent trust, and outlines future challenges in evaluation and compositional defenses.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a robot that does not just follow a rigid list of commands, but instead understands the world through language, vision, and experience. These machines, often called embodied agents, are changing how we think about automation. Instead of being programmed with specific steps for every possible situation, they use powerful foundation models—systems trained on vast amounts of human knowledge and visual data—to perceive their surroundings, reason about what to do, and decide how to move their own bodies. A robot might look at a cluttered table, understand that a cup is in danger of falling, and then plan a sequence of movements to save it, all without a human typing out a single instruction. This shift from fixed programming to flexible, intelligent decision-making promises to bring robots into our homes, hospitals, and factories. However, this new freedom brings a new kind of risk. When a machine's decisions are driven by information it reads, sees, or remembers, that information becomes a potential point of failure. If an attacker can trick the robot's understanding of the world, they can make the machine act in ways that are dangerous, even if the robot's physical motors and gears are perfectly secure.
A team of researchers at Wuhan University has mapped out exactly where these dangers lie. They studied 58 different ways to attack these intelligent robots and 61 different ways to defend them, creating a comprehensive guide to the security of this emerging technology. Their work reveals that the old ways of thinking about robot safety are no longer enough. In the past, securing a robot meant protecting its wires and its software code from hackers. Today, the most vulnerable parts are the robot's senses and its mind. The researchers found that attackers do not need to break into a robot's operating system to cause harm; they can simply change what the robot sees or what it is told to do. By organizing these threats into a clear structure, the team showed that the most common attacks target the robot's eyes and its hands, while the most common defenses are placed right before the robot moves. This imbalance leaves many other critical areas, such as the robot's long-term memory and its communication with other machines, largely unprotected.
The researchers began by defining the different layers of a robot's life, from the moment its software is created to the moment it physically interacts with the world. They identified twelve specific entry points where an attacker could first introduce a lie or a trick. These range from the very beginning, where the robot's learning data might be poisoned before it is even deployed, to the very end, where the robot's physical movements are directly hijacked. A key insight from their work is that the method of attack is not the same as the place it enters. For example, a "backdoor" is a type of hidden trap that can be planted in the robot's training data, but it might only be triggered later by a specific phrase spoken by a user or a specific image shown to the robot. The researchers argue that security experts must focus on where the attack first crosses the line into the robot's trusted world, rather than just the technique used to cross it. This distinction helps in understanding how a small lie in a robot's memory can eventually lead to a large physical mistake.
When the team analyzed the landscape of current research, a clear pattern emerged. The majority of attack studies focused on two specific areas: the robot's multimodal perception and its action interfaces. In plain terms, this means most researchers are trying to trick the robot's eyes or force it to grab the wrong object. They found that half of the recorded attacks targeted the robot's sensors, such as cameras or microphones, or the signals that tell the robot how to move. This makes sense because these are the direct links between the digital mind and the physical body. If an attacker can fool the camera into seeing a stop sign as a go sign, or trick the robot into thinking a wall is open space, the consequences can be immediate and physical. Similarly, many attacks target the final step where the robot decides on a specific movement, such as a command to turn a motor or lift a weight.
However, the defenses tell a different story. The researchers discovered that most security measures are concentrated at the very end of the process, right before the robot executes an action. This is like having a guard who only checks the final product before it leaves the factory, rather than checking the raw materials or the design plans. While this "runtime" protection is important, it leaves the earlier stages of the robot's thinking process exposed. The study showed that defenses for the robot's long-term memory, its communication with other robots, and the integrity of its internal map of the world are surprisingly rare. For instance, there are very few studies on how to protect a robot's memory from being poisoned. If a robot learns a false fact from a malicious source and stores it, that lie can influence its decisions for a long time, causing repeated errors that a simple action-checker might miss.
The team also highlighted the unique challenges of testing these systems. Unlike a text-based chatbot that only needs to be judged on its words, an embodied robot must be tested on its physical actions. A robot might successfully follow a malicious instruction to "move the cup," but if it drops the cup or knocks over a vase while doing so, the attack has succeeded in a way that a text-only test would never catch. The researchers noted that many current studies rely on computer simulations, which are useful but often fail to capture the messy reality of the physical world, such as changes in lighting, camera angles, or the unpredictable nature of real objects. They argued that true security requires testing robots in real environments, where the consequences of a mistake are tangible. They also pointed out that as robots begin to work together in groups, the risk grows. If one robot in a team is compromised, it can spread bad information to the others, causing the entire group to fail.
Looking ahead, the researchers suggest that the field needs to move beyond simple fixes. They propose that future robots should be designed with a deeper understanding of trust. This means that every piece of information a robot receives—from a human voice, a camera image, or a message from another robot—should carry a label indicating its source and its authority. A robot should know that a sign on a wall is just a description of the world and not a command to change its behavior, while a direct order from a human operator should be treated with higher priority. They also emphasize the need for better ways to verify the robot's internal state. If a robot believes a door is open, there should be a way to check if that belief is based on solid evidence or a trick. The goal is to build systems where safety is woven into every layer, from the data used to train the robot to the final movement of its arm.
The work of Liu and his colleagues serves as a crucial roadmap for a rapidly evolving field. By separating the entry point of an attack from the method used, they have provided a clearer way to understand the vulnerabilities of intelligent machines. Their findings suggest that while we are getting better at stopping robots from doing the wrong thing at the very last second, we are not yet good at preventing them from being tricked into thinking the wrong thing in the first place. As these machines become more integrated into our daily lives, the challenge will be to ensure that their ability to learn and adapt does not come at the cost of their safety. The path forward requires a shift from patching individual holes to building a system where trust is verified at every step, ensuring that the robot's actions remain aligned with human intent, even in the face of sophisticated deception.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.