Dynin-Robotics: Omnimodal Unified Diffusion Vision-Language-Action Model
Dynin-Robotics introduces an omnimodal unified diffusion model that treats language, vision, and actions as discrete tokens to jointly learn goal prediction, dynamics modeling, and action generation, achieving competitive performance on multiple benchmarks through shared trajectory modeling and test-time scaling.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Robots have long struggled to understand the world the way humans do. While a machine can be programmed to move its arm to a specific coordinate, it often fails when the task requires understanding a vague instruction like "hand me something to cut the package." To solve this, researchers are building systems that connect language, vision, and physical movement into a single brain. These systems, known as vision-language-action models, try to learn not just what an object is, but how it behaves and how to manipulate it. The challenge has been that most current approaches treat these skills separately: one part of the system might understand the words, another might recognize the picture, and a third might calculate the motor commands. This separation often leaves the robot confused when the real world changes or when the instructions are phrased in unexpected ways.
A team of researchers at Seoul National University has introduced a new approach called Dynin-Robotics that unifies these separate skills into one cohesive model. Instead of building a robot brain with different modules for different tasks, they created a single system that learns to predict the future in multiple ways at once. Imagine a robot that, when given a task, does not just calculate the next move, but also imagines what the scene will look like after that move, what the final goal state should be, and how to describe the task in words. By training a single model to handle all these predictions simultaneously, the researchers found that the robot becomes much better at following instructions, even when those instructions are phrased differently than it was trained on.
The core of this system is a method called masked diffusion. In simple terms, this is a learning process where the model is shown a sequence of information—such as a video of a task, a written instruction, and the robot's movements—but some parts of that sequence are hidden or "masked." The model's job is to look at the visible parts and guess what the hidden parts should be. For example, it might see a picture of a cup and the words "pick up the cup," but the movement data is hidden, so it must predict the correct motion. In another scenario, it might see the movement and the instruction, but the final picture of the cup in the hand is hidden, so it must predict the outcome. By practicing these different types of guessing games on a massive dataset of over 1.3 million robot trajectories, the model learns a deep understanding of how language, vision, and action are connected.
What makes this approach unique is that the same model can switch between these different roles instantly. It can act as a policy, deciding what action to take next; as a world model, predicting what the camera will see after an action; as a goal predictor, visualizing the final successful state of a task; or as a teacher, reconstructing the original instruction from a video of the robot working. This flexibility allows the system to use its own predictions to improve its performance. For instance, before the robot decides how to move its arm, it can first predict what the goal state looks like. It then uses that predicted goal as a guide to refine its action plan. This process of looking ahead and adjusting the plan in real time helps the robot handle complex tasks that require precise alignment, such as inserting a flower into a vase or stacking blocks in a specific order.
The researchers tested this system on a variety of benchmarks to see how well it performed compared to other leading robot models. On standard simulation tasks, the model achieved a 98.1% success rate, placing it among the top performers in the field. More importantly, when tested on tasks where the instructions were changed to be more indirect or abstract, the system showed a significant ability to adapt. In one set of experiments, the model was tested on a task where it had to select a fruit based on a description like "something naturally sweet and juicy" rather than a direct command like "pick up the apple." While other models struggled with these shifts in language, Dynin-Robotics maintained a high success rate, demonstrating that it had learned the underlying concept of the task rather than just memorizing specific phrases.
The system also proved effective in the real world. The researchers deployed the model on a physical robot arm known as the Franka Research 3. In a series of real-world tests involving picking up fruit, sorting colored cubes, and stacking them in specific orders, the robot achieved an average success rate of 78.4% across four different manipulation conditions. This performance was particularly strong in tasks that required following a sequence of steps, such as stacking cubes in a color order specified by the user. The ability to visualize the goal and the intermediate steps allowed the robot to correct its course if it made a mistake, a capability that is often missing in simpler systems.
Beyond its ability to control a robot, the model demonstrated a surprising capacity for understanding and generating descriptions. When shown a video of a robot performing a task, the system could accurately describe what was happening, using verbs like "pick," "place," and "fold," and identifying objects by their color and location. Conversely, when given a task description, it could generate a visual prediction of what the final scene would look like. These capabilities suggest that the model has built a rich, internal representation of the physical world that goes beyond simple motor control.
To make this system fast enough for real-time use, the researchers also developed a specialized method for running the model. Because the model works by iteratively refining its guesses, it can be slow if it has to process every single step one by one. The team created a technique that allows the model to predict and commit to multiple steps of movement at the same time. This optimization sped up the robot's decision-making process by nearly 30 times compared to the standard method, making it feasible to run the complex model on hardware that can keep up with the speed of a physical robot.
The findings of this work suggest that treating robot learning as a unified prediction problem is a powerful strategy. By combining the ability to understand language, predict visual changes, and generate actions into a single framework, the researchers have created a system that is more robust and adaptable than previous models. The success of Dynin-Robotics indicates that when a robot can imagine the future and understand the context of a task, it becomes much more capable of handling the unpredictable nature of the real world. While there is still work to be done, particularly in extending these capabilities to longer and more complex sequences of actions, this approach offers a promising path toward robots that can truly understand and interact with their environment.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.