RoboShape: Information-Theoretic Point Cloud Representations for Privacy-Aware Robot Perception

Robots are increasingly moving out of laboratories and into our homes, hospitals, and workplaces. To navigate these complex spaces, they rely on sensors that capture the world as a cloud of millions of tiny dots, creating a three-dimensional map of their surroundings. These maps allow machines to recognize a chair, avoid a table, or plan a path through a crowded room. However, as robots share these maps with cloud servers or other machines to learn from one another, a new problem has emerged. The very richness that makes these maps useful for navigation also reveals intimate details about the people who live there. A scan of a living room might show a sofa, but the arrangement of furniture and the layout of the space can also reveal whether a room is a bedroom, a home office, or a private medical area—information the occupants never intended to share.

For years, engineers have struggled with this tension between utility and privacy. Traditional methods to protect data often act like a blunt instrument. Adding random noise to a map might hide the room type, but it also blurs the chair and the door, making the robot less effective. Other approaches shuffle the data or try to guess what to hide, but they lack a precise way to measure exactly how much private information remains. The core challenge has been finding a way to keep the details a robot needs to do its job while surgically removing the details that reveal a person's private life, without degrading the overall quality of the map.

A team of researchers has introduced a new system called RoboShape to solve this problem. Instead of treating the entire 3D map as a single, indivisible block of data, they break it down into small, manageable chunks called voxels. Think of a voxel as a tiny, three-dimensional pixel that represents a small cube of space. The researchers start with a powerful, pre-trained computer program that has already learned to understand these chunks well. This program converts each chunk of the 3D map into a long list of numbers, known as an embedding, which captures everything the robot knows about that specific spot, including both the object it sees and the type of room it is in.

The innovation lies in what happens next. The researchers added a lightweight, trainable layer on top of this existing program. This new layer acts like a filter that reshapes the long list of numbers into a much shorter list. The goal of this reshaping is guided by a mathematical principle called mutual information, which essentially measures how much one piece of data tells you about another. The system is trained with two opposing instructions. First, it is told to keep as much information as possible about the object itself, ensuring the robot can still recognize a chair or a desk. Second, it is told to remove as much information as possible about the room type, ensuring the robot cannot tell if it is in a bedroom or a bathroom.

To teach the system how to balance these competing goals, the researchers used a specific method to estimate the flow of information during training. They treated each small chunk of the map as an independent sample, allowing them to measure precisely how much the shortened list of numbers still revealed about the room type. By adjusting the filter to maximize the connection to the object while minimizing the connection to the room, the system learned to compress the data significantly. The result is a representation that is 87.5% smaller than the original, making it much cheaper and faster to transmit over a network.

The researchers tested this approach on three different real-world datasets containing thousands of indoor scenes, ranging from single rooms to entire buildings. They compared their method against standard techniques that simply add noise or use random compression. The results showed that the new system retained 98.7% of its ability to identify objects, meaning the robot could still perform its tasks almost as well as before. At the same time, the system reduced the ability to guess the room type by 39.3%. In a test where a standard computer program could almost perfectly identify the room type from the original data, the new compressed data made the guess no better than random chance.

To ensure these findings held up in a real-world scenario, the researchers also simulated a robot navigating a virtual environment. They trained a robot to find a wall, a task that requires understanding the geometry of the space. The robot using the new compressed data performed nearly as well as one using the full, uncompressed data. However, when they tested whether the robot could learn to navigate specifically based on the room type—such as moving to a different zone if it was in a bedroom versus a living room—the robot with the new data failed to learn this pattern. It could still find its way around the furniture, but it had lost the ability to recognize the nature of the space it was in.

This work demonstrates that it is possible to design robot perception systems that are both compact and privacy-aware. By using information theory to guide the compression process, the researchers created a tool that allows robots to share what they see without revealing where they are. The system is designed to work with any existing 3D mapping technology, offering a practical way to build perception pipelines that are ready for deployment in human environments. The researchers have released their code and models to the public, providing the robotics community with a method to build machines that respect the privacy of the spaces they enter while remaining capable of performing their duties.