Measuring Language Transfer in Robot Policies: Adding Greek to a Cosmos3 Vision-Language-Action Policy

Robots that can see, understand, and move are becoming a reality, but they currently speak only one language: English. These machines learn by watching videos of humans performing tasks, where the instructions are written in English. The software that helps them understand these words is trained on massive amounts of English text, making it incredibly good at that specific language. For anyone who speaks Greek, or any other language, this creates a hard barrier. There are no video libraries of robots moving objects with Greek instructions, and building one from scratch would require months of manually guiding a robot arm for every single action. The question researchers asked was simple: could they take the existing English data, translate the instructions into Greek using a computer, and teach the robot to understand the new language without rebuilding the entire system?

The answer turned out to be more complicated than a simple translation. A team of researchers at Sophea AI and KIEFER SA in Athens took an open-source robot system and fed it thousands of Greek instructions generated by a machine. They did not change the robot's brain or its physical structure; they simply swapped the English text for Greek text. The translation process was fast and easy, taking only hours. However, the real work began when they tried to measure if the robot actually understood what it was being told. In the world of robotics, it is surprisingly easy to trick a system into appearing successful when it is actually just guessing. The researchers found that standard tests, which usually give a clear pass or fail, were completely fooled. They discovered that a robot could follow a Greek command with high success rates even when the command was nonsense, or even when the robot had never been taught Greek at all. This happened because the robot was learning to recognize the scene and the objects rather than reading the words.

To solve this, the team had to invent new ways to test the robot. They built a control group where they deliberately gave the robot wrong instructions in Greek to see if it would still try to do the task. They found that on a small set of ten tasks, a robot trained with machine-translated Greek instructions seemed to succeed about eighty-four percent of the time. But when they tested it with the wrong instructions, it still succeeded about eighty-two percent of the time. This proved the robot was not reading the Greek; it was just reacting to the visual setup. The robot was ignoring the language entirely.

The researchers then tried a different approach. They used a larger set of ninety tasks where the robot had to choose between multiple possible goals in the same scene. Here, the language became the only signal to tell the robot what to do. They found that a robot trained on both English and Greek instructions could follow the Greek commands about twenty-seven percent of the time, which was significantly better than random chance. However, a robot trained only on Greek instructions, without any English data to support it, barely improved over random guessing. While bilingual training performed better on average than Greek-only training, the statistical ranges of their performance overlapped significantly, meaning the data could not definitively confirm that English training acted as a necessary scaffold for the Greek learning. The most robust finding was that target-language demonstrations are necessary, but on their own, they are often insufficient; a policy trained only on Greek barely follows the language, staying within three points of its own failure floor across multiple test runs.

The study also uncovered that the robot was not learning the Greek language in the way a human would. Instead, it was memorizing the specific phrasing used by the machine translator. When the researchers tested the robot with Greek sentences written by a different computer program, the robot's performance dropped sharply. It had learned the style of the first translator, not the language itself. To fix this, they taught the robot the same task using seven different Greek phrasings. This simple change made the robot much more robust, cutting the performance drop in half when tested on new phrasings. This showed that variety in training data is essential for the robot to generalize beyond a single machine's writing style.

Perhaps the most surprising finding was that some common-sense improvements actually made the robot worse. The team tried starting the robot's training with a model that had already been adapted to Greek, thinking this would give it a head start. Instead, the robot performed worse in both English and Greek. They also tried to let the part of the robot's brain that processes language learn freely, rather than keeping it frozen. This also degraded performance. The results suggest that for these specific systems, the best strategy is to keep the language understanding part of the brain exactly as it was trained, and only teach the robot how to move using the new language instructions.

The researchers also attempted to test these methods on a massive collection of real-world robot data, far larger than their simulated tests. They translated over fifty thousand real robot episodes into Greek. However, they could not measure the success of this data because they lacked the physical robot hardware required to run the tests. This highlighted a major bottleneck in the field: translating the data is cheap and fast, but verifying that the robot actually learned is slow, expensive, and requires physical equipment.

Ultimately, the paper concludes that adding a new language to a robot is possible, but it is not as simple as translation. It requires a specific type of base model that already understands the language, a mix of training data in both the new language and English, and a very careful testing process that includes deliberate failures to ensure the robot is actually listening. The robot does not learn the language in a deep, human way; it learns to associate specific patterns of words with actions, and it needs the stability of English training to do so, though the exact mechanism of how English supports Greek remains an open question rather than a confirmed rule. The work serves as a warning to others in the field: do not trust a robot's success rate at face value, and always test with a control group that proves the robot is not just guessing. The path to a multilingual robot is paved not just with data, but with the rigorous discipline of proving that the data was actually understood.