VirTooS: A ROS 2 - Unity Virtualization Toolkit for Fleet Management of Autonomous Mobile Robots
This paper introduces VirTooS, a Python/C# toolkit that integrates ROS 2 and Unity to enable scalable, mixed-reality fleet management simulations for autonomous mobile robots, facilitating seamless interaction between real and virtual agents through customizable scenarios and containerized deployment.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a world where a team of robots must work together to complete a complex job, such as delivering packages in a busy warehouse or exploring a dangerous cave. In the real world, testing how these machines coordinate is expensive, risky, and often impossible to repeat if something goes wrong. To solve this, scientists have long turned to computer simulations, creating digital twins of robots and their surroundings to test ideas safely. However, a gap has existed between the simple, blocky simulations used by engineers and the rich, realistic environments needed to truly understand how robots will behave when they meet the messy reality of the physical world. Researchers needed a way to blend the two, allowing a real robot to walk through a room while seeing and reacting to a digital world that doesn't physically exist, all while working alongside virtual teammates.
This is the challenge addressed by a new toolkit called VIRTOOS, developed by a team of researchers at the University of Bologna. The system acts as a bridge, connecting two powerful pieces of software: ROS 2, a standard framework that helps robots think and move, and Unity, a game engine famous for creating stunningly realistic 3D worlds. By linking these two, the researchers created a mixed-reality environment where physical robots and digital robots can share the same space, see the same obstacles, and cooperate on tasks as if they were all real. The goal was not just to watch a simulation, but to build a flexible system where engineers can design complex scenarios, test how groups of robots make decisions together, and validate their strategies before ever deploying them in the real world.
The core of VIRTOOS is a modular design that breaks the problem into three distinct layers, each handling a specific part of the robot's life. The first layer is the "brain" of the fleet, a management system that decides who does what. Instead of a single computer telling every robot what to do, this system uses distributed optimization, a method where each robot figures out its own best move while talking to its neighbors. This allows the group to adapt quickly if a new task appears or if a robot breaks down. The second layer handles the actual movement. It takes the high-level decisions from the first layer and translates them into specific commands for the wheels and sensors, ensuring the robot can navigate around obstacles and avoid crashing into its teammates. The final layer is the visual interface, built inside the Unity game engine. This is where the magic of the mixed reality happens. The researchers built tools to generate virtual indoor environments, complete with walls, shelves, and lighting, which can be customized instantly. They also integrated real-world robot models into this digital space, ensuring that the virtual robot looks and moves exactly like its physical counterpart.
To make this work, the team had to solve a tricky problem of perspective. The software that runs the robots and the software that draws the 3D world speak different languages regarding how they define space. One system might say "up" is the ceiling, while the other says "up" is the side of a wall. The researchers wrote custom scripts to translate these directions instantly, so that when a real robot moves forward, its digital twin moves forward in the simulation, and when the virtual robot sees a wall, the real robot receives a signal to stop. This connection is maintained through a high-speed network that allows data to flow back and forth in real time. The system is also designed to be portable; the entire software suite is packaged in a container, a self-contained unit that can be run on almost any computer, making it easy for other researchers to adopt and test their own ideas without needing a supercomputer.
The researchers tested their toolkit with three different scenarios to prove it worked. In the first, they created a fully virtual warehouse filled with shelves and obstacles. They sent a fleet of four digital robots into this space to solve a dynamic task assignment problem. This means the robots had to decide among themselves who would go to which location to pick up items, constantly re-evaluating their plan as new tasks appeared. The simulation showed that the robots could successfully map the environment, avoid collisions with each other and the shelves, and complete their missions. In the second scenario, they took a real robot, a Clearpath Jackal, and placed it in their laboratory. While the robot moved on the actual floor, its sensors were fed data from a virtual environment built in Unity. The robot could "see" virtual walls and obstacles that did not exist in the room, and it successfully navigated around them as if they were real. This demonstrated that a physical machine could safely interact with a digital world.
The most ambitious test combined these two ideas. The researchers set up a mixed-reality experiment where one real robot and three virtual robots had to work together in the same space. The real robot moved around a small room in the lab, while the other three existed only on a computer screen. They were all given a shared goal: to visit a series of locations in a coordinated way. The real robot had to avoid the virtual robots, which were moving around it in the simulation, just as if they were physical machines. The system worked seamlessly. The real robot received data about the virtual obstacles and adjusted its path to avoid them, while the virtual robots reacted to the presence of the real one. At one point, the robots had to pause their movement because the space was too tight to pass safely, a behavior that emerged naturally from their shared understanding of the environment. Once the group re-calculated their plan, they resumed their journey.
The results show that VIRTOOS is a viable tool for bridging the gap between algorithmic research and real-world deployment. It allows scientists to test complex coordination strategies in a safe, controlled setting that feels real. The system is not limited to simple navigation; it can be used to test how robots map unknown areas, how they deliver packages, or how they might work alongside humans in crowded spaces. Because the toolkit is open-source and designed to be easily modified, it offers a new way for the robotics community to validate their ideas. By letting real and virtual robots share the same world, researchers can now explore the future of multi-robot teams with a level of confidence and realism that was previously out of reach.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.