Interaction Scaling: Grounding the Third Axis of Test-Time Compute
This paper introduces "interaction scaling" as a distinct third axis of test-time compute that surpasses the limits of reasoning and sampling by grounding both feedback and evaluation in real-world observations, thereby enabling continuous improvement on hard tasks where traditional methods plateau.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you have a super-smart robot artist who can draw pictures, write code, or design slides. You want it to make the perfect version of a project. Usually, when the robot messes up, we try to fix it in two ways:
- Think Harder: We tell the robot, "Take a deep breath and think about this for a longer time before you draw."
- Try More Times: We tell the robot, "Draw 100 different versions, and we'll pick the best one."
The paper argues that both of these methods hit a glass ceiling. Why? Because the robot is only using its own brain (its "frozen weights") and the same original instructions. It's like asking a person to find a mistake in their own drawing just by staring at it harder or drawing it 100 times on the same piece of paper. They can't see what they don't already know. They are stuck in a closed loop.
The Third Way: The "Real-World" Test
The paper introduces a third way called Interaction Scaling. Instead of just thinking or trying again, the robot actually does the thing and then gets a report from a real instrument.
Think of it like this:
- The Old Way: The robot draws a slide, looks at it, and says, "Hmm, maybe the text is too big?" It's just guessing based on its own opinion.
- The New Way (Grounded Interaction): The robot draws the slide, then hands it to a measuring tape robot (an instrument). The measuring tape robot doesn't have opinions; it just measures. It says, "Hey, this text is 5 pixels too big and it's spilling off the edge of the paper." The drawing robot hears this real fact, fixes the specific error, and tries again.
This is the magic: The measuring tape brings in new information that the drawing robot didn't have before. It breaks the glass ceiling.
The Big Trap: The "Blind Judge"
Here is the most important part of the story, and it's a bit of a plot twist. The paper says this new method only works if both the feedback and the score are "grounded" (based on real measurements).
Imagine you are trying to fix a slide deck.
- The Grounded Instrument: A tool that measures the actual layout on the computer screen. It sees that text is overlapping.
- The "Blind" Judge: A different AI that looks at a screenshot of the slide.
The paper found a hilarious but serious problem: Screenshots are liars.
When you take a screenshot of a slide where text is spilling off the edge, the camera (the screenshot) cuts off the edge. The text that is broken is literally cropped out of the picture!
- The Measuring Tape sees the mess and says, "Fix it!"
- The Screenshot Judge looks at the cropped image, sees nothing wrong, and says, "Perfect! 10/10!"
If you use the Screenshot Judge to tell the robot what to fix, the robot actually makes the slides worse. It tries to fix things that aren't broken in the picture, while leaving the real broken parts alone. The paper showed that on 15 tricky academic figures, the Screenshot Judge rated 14 out of 15 broken images as "perfect," while the measuring tool found only 3 were actually clean.
The Results: What Actually Happened?
The researchers tested this on seven different types of tasks (code, slides, web pages, etc.) using a "Proposer-Reviewer" loop. Here is what they found:
- For Code: When they let the robot run its code and read the actual error messages (the "grounded" feedback), they reached a 100% pass rate on hard tasks. The old methods (thinking longer or picking the best of 100) got stuck at around 66% to 86%, even if they had a perfect "oracle" to pick the best try. The interaction loop kept climbing while the others hit a wall.
- For Visuals (Slides & Figures): When they used the measuring tool to find defects, they removed 40% to 74% of the layout errors.
- On Slides, the measuring tool fixed 73% of the defects.
- On Academic Figures, it fixed 74%.
- On Web Pages, it fixed 47%.
- On Animations, it fixed 40%.
- The "Blind" Judge Failure: When they used the Screenshot Judge to review the slides, the number of defects actually went up (by about 2.44 defects per slide on average). The robot got confused by the blind judge and broke things that were fine.
What About Other Models?
This isn't just about one specific robot. They tested this on three different families of AI models (Sonnet 4, Qwen3-235B, and GPT-5). In every single case, the "interaction loop" with real measurements fixed the errors, while the "thinking harder" methods hit a limit. The loop reached 100% pass rate with zero variation, meaning it worked perfectly every single time, no matter how the robot started.
The "Student" Lesson
The paper also tried to teach a smaller, cheaper robot (an 8-billion-parameter model) to do this by showing it the "teacher" robot's successful fixes.
- The small student could do about 51% of the teacher's work with just one try.
- If they let the small student try two times, it could do 70% of the teacher's work.
- This suggests that the "magic" of the loop can be learned, but the loop itself (the real-world measurement) is still needed to get the best results.
The Bottom Line
The paper concludes that simply "thinking longer" or "trying more times" isn't enough because the robot is stuck inside its own head. To get better, the robot needs to step outside, build something, and have a real measuring tool tell it exactly what's wrong.
But there's a catch: You can't use a "blind" judge (like a screenshot reader) to measure the success. If the judge can't see the mistake (because it's cropped out or too small), the robot will never learn to fix it. The paper proves that grounding (using real measurements for both feedback and scoring) is the secret sauce that makes the whole system work. Without it, the robot is just spinning its wheels.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.