GRAFF-MM : Graph-Frequency Filtering for Multi-Modal Memorability Prediction

Some images stick in the mind while others fade almost instantly. This quality, known as memorability, is not simply about how pretty a picture is or how much a person likes it. Instead, it is a measurable trait of the image itself, a property that determines how likely a person is to recall that specific scene after seeing it once. Researchers have long known that human intuition is poor at guessing which images will be remembered; we often assume a striking face or a bright color will make an image memorable, only to find that a mundane scene with a specific arrangement of objects is what actually lingers. To understand this phenomenon, scientists have turned to computers, building models that try to predict these scores by analyzing the visual ingredients of a photograph. Early attempts relied on simple measurements of color and shape, but these often missed the deeper context that makes an image memorable. More recent efforts have used complex artificial intelligence to look at objects and their relationships, yet many of these systems still overlook the subtle, high-speed details of a scene or the way a human eye naturally scans a picture.

A team of researchers has now proposed a new approach that brings together several different ways of seeing an image to solve this puzzle. Their work, called GRAFF-MM, operates on the idea that no single method can capture everything that makes an image memorable. Instead, the model combines three distinct streams of information: the spatial layout of objects, the areas where a human eye is most likely to focus, and the descriptive language that explains what is happening in the scene. To handle the spatial layout, the researchers built a system that treats an image like a map of connected points, where each point represents an object and the lines between them show how those objects relate to one another. Crucially, they improved this system by teaching it to pay attention to both the broad, smooth patterns of the scene and the sharp, detailed edges that often signal something unusual or important. This dual focus allows the computer to see not just that a motorcycle is on a road, but that the motorcycle is leaning sharply into a turn, a detail that might be the key to why the image is remembered.

The researchers tested this new model against four different collections of images, ranging from thousands of diverse photographs to smaller sets focused on specific objects. In every case, the new model outperformed previous methods, achieving a level of accuracy that closely matches how humans agree on which images are memorable. On one major dataset, the model reached a score of 0.692, and on another, it reached 0.711, numbers that indicate a strong alignment with human memory. The study also included a detailed look at why the model works, showing that removing any one of its three main parts—the spatial map, the eye-tracking guide, or the text description—caused the performance to drop. This confirms that each piece adds something unique: the text provides the story, the eye-tracking guide shows where the action is, and the spatial map reveals the structure.

One of the most significant findings is that the model's ability to detect high-frequency details—those quick, sharp changes in the image that earlier systems often smoothed over—was essential for its success. By filtering the visual data to keep both the big picture and the fine details, the model could spot the unusual arrangements or specific actions that make an image stand out. The researchers also visualized how the model learned, showing that as it combined these different types of information, the images it analyzed began to group together more clearly based on how memorable they were. While the model is not perfect and sometimes struggles with blurry images or overly complex scenes, it represents a significant step forward in understanding how computers can learn to see the world the way human memory does. The work suggests that to truly predict what we will remember, we must look at an image not just as a collection of pixels, but as a story told through space, attention, and language.