Chain of Spatial Thoughts: Modality-Agnostic Spatial Grounding for Vision Language Models

Imagine you are teaching a robot to navigate a messy living room. You show it a picture of a couch, a coffee table, and a cat. A basic robot might say, "I see a couch." But to actually pick up a toy without knocking the table over, the robot needs to know much more: How big is the couch? How far is it from the wall? Is the cat sitting on the table or under it? This is the difference between just seeing and truly understanding space. In the world of artificial intelligence, this is called "spatial intelligence." For a long time, computers were great at recognizing objects but terrible at figuring out how those objects fit together in 3D space. To fix this, scientists have tried two main things: either making the computer's brain (the model) huge and expensive, or adding special, heavy-duty tools just for measuring distances. Both work, but they are slow and clunky.

Now, a team of researchers has come up with a clever new trick called "Space Tokens." Think of a standard AI model like a student taking a test. Usually, if the student needs to know how to measure a room, they have to bring in a protractor and a tape measure (extra tools) or memorize a massive encyclopedia of geometry (a huge brain). This new paper suggests a different approach: what if the student could just think in measurements? The researchers found a way to teach the AI to create invisible "mental notes" inside its own thinking process. These notes, or "tokens," act like tiny, continuous whispers of geometry that the AI can use to reason about size, distance, and shape without needing any extra tools or a bigger brain. It's like giving the AI a built-in sense of space that it can use while it talks, making it smarter about the physical world without slowing it down.

The Problem: Big Brains vs. Heavy Tools

Imagine you have a very smart friend who is great at describing pictures. You show them a photo of a kitchen, and they can tell you, "That's a fridge, and that's a toaster." But if you ask, "Can I fit a giant pizza box between the fridge and the toaster?" they might guess wrong because they don't really "feel" the space.

To fix this, scientists have tried two main paths. The first is "scaling," which is like telling the friend to read every book in the library and memorize every possible room layout. This works, but it requires a massive brain that takes forever to run. The second path is "specialized tools," which is like giving the friend a laser measure and a 3D scanner to use every time they look at a picture. This is accurate, but it's slow, expensive, and the friend can't use it if they are in a hurry or if the tools aren't available.

The authors of this paper asked a simple question: Can we teach the friend to just "know" the space in their head, without needing the library or the laser measure?

The Solution: Space Tokens

The researchers introduced a method called Space Tokens. Instead of adding new hardware or making the AI model huge, they taught the AI to generate special "tokens" (which are just tiny pieces of data) that represent 3D space.

Think of these tokens like invisible sticky notes that the AI sticks onto its own thoughts.

1. Scene-Level Notes: Some notes describe the whole room. They capture the shape of the floor, the position of the walls, and the depth of the room.
2. Object-Level Notes: Other notes describe specific items. They hold information about where an object is, how big it is, and which way it is facing.

The magic is that these notes are "continuous," meaning they aren't just simple labels like "big" or "small." They are like precise coordinates and measurements that the AI can use to do math in its head. The AI learns to create these notes by looking at examples from a "teacher" model (a very smart 3D reconstruction system) and then practicing until it can do it on its own.

How They Taught the AI

The training happened in three stages, like a video game with three levels:

• Level 1: Learning the Language of Space. The AI was shown pictures and taught to generate the "sticky notes" (tokens) that match the 3D geometry of the scene. It learned to align its internal thoughts with the precise 3D shapes of the world.
• Level 2: Thinking with the Notes. Once the AI knew how to make the notes, they taught it to use them to answer questions. They forced the AI to look at the notes and say, "Because the fridge is 2 meters away, the pizza box won't fit." This is called "Chain-of-Thought" reasoning, but now the thoughts include spatial data.
• Level 3: Getting Better with Practice. Finally, they used a technique called Reinforcement Learning. The AI tried to answer questions, and if it got the size or distance right, it got a "reward." If it was wrong, it learned from the mistake. This helped the AI get even sharper at using its spatial notes.

What They Found

The results were impressive. The researchers tested their new method on a benchmark called VSI-Bench, which is a tough test for spatial understanding.

• When they applied this to a model called Qwen3-VL-8B, the score went up by 4.3%.
• When they applied it to a stronger model called SenseNova-SI-1.3, the score went up by 1.3%.

But the real win was in specific tasks. The new method became the best in the world at guessing object size (getting 79.2% right) and room size (getting 75.7% right). These are tasks that usually require heavy 3D tools, but this method did it just by using its internal "sticky notes."

Why This Matters

The most exciting part is that the AI didn't need to change its brain or carry extra tools. It just learned to think differently. The researchers also showed that these "sticky notes" are real. They could decode them back into 3D shapes and see that the AI had actually learned the geometry of the room. It wasn't just guessing; it had a genuine understanding of space.

This suggests that we don't need to build bigger, slower, or more expensive AI models to make them spatially smart. We just need to teach them how to create and use these internal spatial notes. It's a lighter, faster, and more flexible way to give machines the ability to navigate our 3D world, which is a huge step forward for things like robots that need to move around our houses or cars that need to drive safely.

In short, the paper shows that by giving AI a way to "think in 3D" using simple, internal tokens, we can make it much better at understanding the physical world without making it heavier or slower. It's a small change in how the AI thinks that leads to a big jump in what it can do.