Optimizing MR-based gaze-decoding for eyes-closed eye-tracking in fMRI

In the quiet hum of an MRI scanner, researchers often ask people to lie perfectly still while their brains light up in response to pictures, sounds, or thoughts. For decades, scientists have known that where a person looks is just as important as what they are thinking. If a participant's eyes drift even slightly, the brain activity recorded can change completely, turning a clear signal into confusing noise. To prevent this, most studies use high-speed cameras to track the eyes, ensuring the person is looking exactly where they should. But these cameras have a blind spot: they cannot see when the eyes are closed. This leaves a vast gap in our understanding. We cannot study what happens in the brain during sleep, during deep relaxation with eyes shut, or when people are asked to imagine scenes in their minds, because we have no way of knowing if their eyes are moving or where they might be pointing.

A team of researchers has now found a way to see through that darkness, not with a camera, but by listening to the magnetic signals of the eyeballs themselves. Using a powerful computer program that learns to recognize patterns in brain scans, they have taught a machine to guess where a person is looking, even when their eyelids are shut tight. The breakthrough does not rely on expensive equipment or perfect conditions. Instead, it relies on a clever training method that allows the system to adapt to different scanners and different tasks, opening a window into the hidden movements of the mind's eye.

The story begins with a tool called DeepMReye, a sophisticated computer program that was already known to work well when people were looking at screens. This program acts like a translator, converting the faint magnetic ripples coming from the eyeballs inside the scanner into a map of where the gaze is directed. However, the original version of this translator was trained mostly on data from people with their eyes wide open. When the researchers tried to use it on people with their eyes closed, the results were shaky. The physical shape of the eye changes slightly when the eyelids press down, and the magnetic signals shift in ways the computer had never seen before. It was like asking a translator who only speaks English to suddenly interpret a language that uses entirely different grammar.

To fix this, the researchers designed a new kind of training session. They brought fifteen volunteers into the scanner and asked them to perform a series of eye movements while their brains were scanned. First, the volunteers looked at a target on a screen, following it as it moved in specific patterns, like tracing triangles in the air with their eyes. This part was done with eyes open, providing the computer with clear, perfect examples of how the magnetic signals look when the eyes are moving in a controlled way. The researchers then used this fresh data to "fine-tune" the computer program, essentially teaching it how their specific scanner and their specific setup saw the world.

The results were immediate and clear. When the researchers tested the newly trained program on the same eye-opening tasks, it performed significantly better than the original version. It tracked the movements with much greater precision, reducing the distance between where the computer thought the eyes were and where they actually were. Crucially, the researchers discovered that this improvement was not just a matter of stretching the numbers to fit a larger screen. The program had genuinely learned the unique way their scanner captured the eye's motion. Even more surprisingly, they found that they did not need the camera data to do this training. They could simply tell the computer, "The person was looking at this specific spot," and the program learned just as well as if it had seen the actual eye movements recorded by a camera. This means that any lab, even one without a camera, could now teach the system to work for them.

The real magic, however, happened when the lights went out. The researchers then asked the volunteers to perform the exact same triangle-tracing task, but this time with their eyes closed, guided only by a sequence of beeps. They also had them blink slowly and keep their eyes open without looking at anything. When they fed the brain scans from these closed-eye sessions into the newly trained program, it successfully reconstructed the eye movements. The computer could "see" the triangles being traced in the dark. To prove this was not a lucky guess, the researchers scrambled the order of the movements in the data. When the sequence was broken, the program could no longer identify the shape, proving it was truly decoding the path of the eye and not just guessing based on the timing of the sounds.

This ability to see closed-eye movements was not just a party trick; it revealed something profound about how the brain and eyes interact. The researchers found that the program could also tell the difference between open and closed eyelids with high accuracy. It could detect when a person was blinking or when their eyes were fully shut, simply by reading the magnetic signals. This suggests that the state of the eyelid leaves a distinct fingerprint on the brain scan, one that the computer can learn to recognize.

Perhaps the most practical discovery was that the program did not need to be trained on closed-eye data to work on closed-eye tasks. The researchers tried training the system using only the open-eye data they had collected earlier. When they tested this version on the closed-eye sessions, it performed almost as well as the version that had been trained on both open and closed data. This means that to study sleep or mental imagery in the future, scientists do not need to struggle to get a camera to work in the dark. They can simply train the system while the subject is awake and looking at a screen, and then apply that knowledge to the closed-eye moments.

The study also addressed a common limitation in how these systems are used. Often, researchers assume that if they just adjust the scale of the output, a generic program will work for any setup. The researchers showed that this is not enough. A simple mathematical stretch of the numbers could not match the performance of a program that had been specifically trained on the local data. The computer needed to learn the specific "voice" of the scanner and the unique way the eyes moved in that particular room.

By combining a simple training routine with a powerful learning algorithm, the researchers have removed a major barrier to studying the brain. They have shown that we can now track where the mind is looking, even when the eyes are shut, without needing a camera to watch them. This opens the door to studying sleep, deep relaxation, and the inner landscapes of imagination with a clarity that was previously impossible. The technology is not perfect; the movements of closed eyes are naturally less precise than open ones, and the computer reflects that reality. But for the first time, the darkness of the closed eye is no longer a blind spot in our understanding of the human brain.