Can Environment-Agnostic Agents Solve ARC-AGI-3 from Warped and Noisy Images?
This paper demonstrates that while environment-agnostic agents struggle to solve ARC-AGI-3 tasks from heavily warped and noisy images, the GPT-6-astra model achieves near-ceiling performance under these extreme conditions, significantly outperforming GPT-5.6-sol by effectively combining image processing with decision-making under uncertainty.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the author. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a computer program that must learn to play a game not by reading rules or seeing a clean grid of numbers, but by looking at a picture of the game board. This is the challenge of building an environment-agnostic agent: a single software mind that can enter different digital worlds, figure out how they work just by looking, and take actions to win, all without being reprogrammed for each new game. The difficulty multiplies when the picture it sees is not clear. What if the image is blurry with static, or if the board itself looks bent, stretched, or twisted as if viewed through a funhouse mirror? In the real world, our eyes and brains are remarkably good at correcting for these distortions, but for a machine, seeing a target that is warped and then figuring out exactly where to click on the original, flat board is a profound test of intelligence. This question sits at the heart of a new study exploring whether artificial intelligence can learn to see clearly through a haze of visual chaos.
Researchers at SingularityNET set out to test this by feeding a specific type of artificial agent a series of logic puzzles known as ARC-AGI-3. These puzzles are designed to test reasoning, requiring the solver to discover hidden rules and patterns. In this experiment, the agent was never given a clean list of numbers or a text description of the board. Instead, it received only images. To make the task brutally difficult, the researchers deliberately corrupted these images. They added random static, similar to the grain seen on an old television screen, and they applied geometric warps that bent the straight lines of the board into curves or shifted the perspective entirely. Crucially, the distortion changed with every single frame the agent saw, meaning the board could look different from one moment to the next, even if the game state had not changed. The agent had to look at a twisted, noisy image, figure out what was happening, and then submit a precise coordinate on the original, unwarped board to make a move. It was like trying to hit a target on a wall while looking at it through a wobbly, dirty window that kept changing shape.
The study compared two different versions of the artificial intelligence, one older and one newer, to see how they handled this visual chaos. The older version, when faced with the twisted images, struggled significantly. Its performance dropped sharply, especially when the images were bent in complex ways. The researchers found that the bending of the image was far more confusing for the machine than the static noise alone. To help the older agent, the scientists introduced a simple tool: a "cursor feedback" feature. This allowed the agent to propose a location on the board and see a picture of where that point would appear on the distorted image, essentially letting it check its guess before committing. This helped the older agent recover some of its lost ability, particularly in games that required clicking, but it could not overcome the most severe distortions. Even with this help, the older agent failed to solve most of the hardest puzzles when the images were both extremely noisy and heavily warped.
The newer artificial intelligence, however, performed with startling resilience. When faced with the same severe distortions that nearly paralyzed the older model, this agent solved almost every puzzle, achieving a success rate of over ninety-five percent even without the cursor feedback tool. It managed to navigate the twisted, noisy images and correctly identify the coordinates on the original board. The researchers pushed this agent even further in follow-up tests, increasing the noise to levels where individual pixels were almost indistinguishable and adding random rotations to the board. Even under these extreme conditions, the agent did not collapse. While its overall success rate dropped, it still managed to solve a significant portion of the games. In one specific game, it achieved a near-perfect score despite the images being so noisy and distorted that they looked almost unrecognizable to a human observer.
Digging into how this agent succeeded, the researchers looked at the files it created during its play. They found that the agent had written its own computer scripts to process the images. It used these tools to clean up the noise and straighten out the warped lines, effectively rebuilding a clear view of the board from the chaotic input before making its decisions. This suggests that the agent did not just "guess" its way through the confusion; it actively constructed a clearer picture of reality using tools it developed on the fly. The study concludes that while severe visual distortion can break simpler models, more advanced agents can learn to compensate for it by building internal models of the game and using image-processing tools to correct their view. The findings suggest that the path to robust artificial intelligence may lie in the ability to not just see the world as it is presented, but to actively work to understand the world behind the distortion.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.