The Impact of Noise on Memorability in Minds and Machines
This paper demonstrates that while conventional deep neural networks fail to match human robustness in estimating image memorability under noise due to their reliance on individual image features, the newly proposed GraphMem model successfully aligns with human performance by leveraging the relational structure among images.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer
We do not live in a perfect visual world. Our eyes often see through rain-streaked windshields, grainy security footage, or the dim light of evening. Yet, despite these imperfections, human memory is remarkably consistent. If you show a large group of people a photograph of a child blowing out birthday candles and another of an empty parking lot, most will remember the party scene and forget the parking lot. This phenomenon, known as image memorability, suggests that some pictures are simply easier for our brains to hold onto than others. Scientists have long believed that memorable images are processed more efficiently by the brain, allowing them to be recognized and stored quickly. To study this, researchers have increasingly turned to artificial intelligence, using deep neural networks—computer systems designed to mimic the human brain—to predict which images people will remember. These models are now used to score and select images for everything from advertising to scientific research, often replacing human observers. But a critical question remains: do these machines understand memory the same way we do, especially when the images they see are degraded by noise or blur?
A team of researchers at George Mason University set out to test this by comparing how both humans and artificial intelligence handle visual noise. They began with a standard set of images known to have varying levels of memorability, ranging from highly memorable scenes to those that are easily forgotten. They then systematically degraded these images by adding different amounts of visual static, similar to the grain seen on old television screens, and also by blurring them. First, they ran these noisy images through several state-of-the-art computer models that are currently considered the best at predicting human memory. The results were surprising. The computer models behaved in a way that resembled a phenomenon called stochastic resonance, where adding a little bit of noise can actually make a weak signal easier to detect. In the models, adding noise made the least memorable images appear more memorable, while making the most memorable images appear less so. It was as if the noise confused the computer's focus, causing it to misjudge the importance of the image.
The researchers then asked human participants to perform the same task, viewing the same noisy and blurred images in a continuous recognition game where they had to identify repeated pictures. The human results told a completely different story. When the images were degraded, human memory for the highly memorable pictures remained strong and resistant to the noise. However, memory for the less memorable pictures declined sharply. Unlike the computers, humans did not find that noise helped them remember the forgettable images; instead, the noise simply made it harder to remember anything, but the brain's ability to hold onto the most striking images was far more robust. The study found that this gap between human and machine performance persisted even when the researchers retrained the computer models on noisy images, a technique that usually helps machines recognize degraded objects better. The models continued to misjudge how noise affects memory, suggesting that their internal way of processing images is fundamentally different from how humans learn and remember.
To understand why the computers were failing, the researchers looked at where the models were "looking" when they made their judgments. They found that for highly memorable images, the models focused their attention on a specific, central part of the picture. When noise was added, this focus scattered, and the model's confidence in the image's memorability dropped. For less memorable images, the models initially had a scattered focus, but the noise sometimes caused them to concentrate on a single spot, accidentally boosting their score. In contrast, human memory does not rely on a single focal point in the same way. The researchers discovered that human memory depends heavily on how an image relates to other images in a set, rather than just the features of the image itself. To test this, they built a new model called GraphMem. Instead of analyzing each image in isolation, this model looks at the relationships between all the images, mapping them based on how similar they are to one another.
When the researchers tested this new approach, the results shifted dramatically. GraphMem, which accounts for the context and relationships between images, performed much more like a human. It correctly predicted that highly memorable images would remain stable even as noise increased, while less memorable ones would suffer. It did not fall into the trap of thinking that noise made bad images better. The study suggests that the reason current artificial intelligence struggles with memory under noisy conditions is that it treats every image as a standalone object, ignoring the broader context in which we naturally view the world. Humans, having grown up seeing the world in all its imperfect clarity, have learned to filter out noise and rely on the distinctiveness of an image relative to everything else we have seen. The findings indicate that to build truly human-like memory systems, artificial intelligence must move beyond analyzing individual features and start understanding the web of relationships that connects one image to the next.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.