ML-Assisted Cognitive Characterization of Human Problem Solving Behaviors through Visuo-Spatial Game
This study utilized an ensemble-based Random Forest classifier and regressor on eye-tracking data from a visuo-spatial game to successfully categorize players into distinct behavioral groups, distinguishing between solvers (experts and persistent players) and non-solvers (triers and quitters) based on their gaze and gameplay patterns.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer
The human mind is a master of navigation, constantly scanning the world to find patterns, solve puzzles, and reach goals. For decades, scientists have tried to understand how we do this by watching what we do: the moves we make, the choices we take, and the strategies we employ. But a significant part of this mental work happens before we even act. It happens in the way our eyes move, darting from one point to another, lingering on details, or scanning the whole picture. These eye movements are not random; they are a direct window into our thinking process. When we look at a problem, our eyes reveal whether we are confused, whether we have found a path forward, or whether we are stuck in a loop of trial and error. By studying these visual habits, researchers can see the invisible architecture of human problem-solving, distinguishing between those who are on the verge of a breakthrough and those who are simply wandering.
A team of researchers at the Indian Institute of Technology Kharagpur decided to put this idea to the test using a classic puzzle and a modern tool. They gathered a group of adults and asked them to solve a specific type of visual puzzle known as an 8-tile arrangement game. In this game, an image is broken into nine blocks, with one block removed (specifically the bottom-right tile), leaving a gap. The player must slide the image tiles into the empty space to reconstruct the original image. It sounds simple, but the path to the solution is rarely straight. The researchers equipped the participants with an eye-tracking device, a camera that records exactly where a person is looking and for how long, capturing thousands of data points per second. As the players moved the tiles, the machine recorded their gaze, creating a detailed map of their attention. The goal was not just to see who solved the puzzle, but to understand how their eyes moved differently depending on whether they were successful, how long they struggled, or if they gave up entirely.
The researchers used a powerful computer program, a type of machine learning model, to analyze the eye-tracking data. They did not look at the final score alone; instead, they watched the story unfold move by move. They discovered that the eyes of the players told a clear story long before the puzzle was finished. The computer could distinguish between two main groups: those who eventually solved the puzzle, whom they called "Solvers," and those who did not, whom they called "Non-solvers." The difference was not in how many moves they made, but in the rhythm and pattern of their gaze. The Solvers showed a distinct visual behavior that the computer could identify with high accuracy, often within the first few moves of the game. This suggested that the way a person looks at a problem reveals their strategy almost immediately.
But the story did not end with just two groups. The researchers found that within the Solvers, there were two very different types of thinkers. The first group, the "Experts," approached the puzzle with a steady, strategic eye. Their gaze was focused and efficient, moving with purpose toward the solution. The second group, the "Persistent players," also solved the puzzle, but their eyes told a different tale. They were erratic, jumping back and forth, revisiting areas, and showing signs of confusion before finally finding their way. They solved the problem, but they did so by wandering through the possibilities rather than seeing the path clearly from the start. Similarly, the Non-solvers were not all the same. Some, the "Triers," spent a long time trying to solve the puzzle. Their eyes showed a mix of effort and exploration; they were looking for the answer, testing ideas, and sometimes even getting close to the solution before giving up. The other group, the "Quitters," showed a lack of engagement. Their gaze patterns indicated they had no real intention of finding the solution; they were not exploring, they were simply waiting for the game to end.
One of the most striking findings was how early these differences appeared. The computer model could predict with great certainty whether a player would succeed or fail after watching only a tiny fraction of the game. This "saddle point," as the researchers called it, is the moment when a player's behavior stabilizes into a clear pattern. For the Experts, this happened almost instantly. For the Triers, it took a little longer as they explored the board, but their pattern of trying was still distinct from the Quitters, who showed no such pattern of effort. The study also looked at how well the computer could predict the future of the game. By watching the early eye movements, the model could estimate how many more moves a player would need to finish or how long they would keep playing before quitting. The predictions were most accurate when the computer was trained on specific types of players, learning the unique "language" of the Experts or the specific struggles of the Triers.
The researchers also looked at the noise in the data to understand the mental state of the players. They found that the Experts had a very clean, steady signal in their eye movements, suggesting a calm and confident mind. The Persistent players, however, had a much noisier signal, with their eyes jumping around more, reflecting the mental effort of figuring things out as they went. The Triers also showed this noisy, fluctuating pattern, which makes sense because they were actively searching for a solution that they could not quite grasp. The Quitters, on the other hand, had a different kind of stability, one born of disengagement rather than focus. This research suggests that the way we look at a problem is a sensitive indicator of our cognitive strategy. It reveals whether we are thinking deeply, whether we are stuck in a loop, or whether we have lost interest.
This study offers a new way to understand human intelligence, not by what people say they are doing, but by what their eyes reveal they are thinking. It shows that problem-solving is not a single skill but a collection of different behaviors, from the strategic clarity of the Expert to the determined but confused search of the Triers. The ability to detect these patterns so early in a task could be valuable for educators or designers who want to help people learn. If a system can recognize that a student is a "Trier" who is on the verge of an insight but needs a small nudge, it could provide the right hint at the right moment to help them succeed. Conversely, it could identify when someone has truly given up, allowing for a different kind of intervention. The work confirms that our eyes are not just passive cameras; they are active participants in our thinking, mapping out the path to a solution before our hands even move.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.