💻 computer science

An extremely coarse feedback signal is sufficient for learning human-aligned visual representations

This study demonstrates that training neural networks with extremely coarse feedback signals, such as distinguishing just eight broad categories, is sufficient to learn visual representations that align more closely with human brain activity and perceptual judgments than models trained with fine-grained or self-supervised objectives.

Original authors: Yash Mehta, Michael F. Bonner

Published 2026-05-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yash Mehta, Michael F. Bonner

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to see the world, just like a human does. For the last decade, scientists have believed that to make a robot's "brain" look like a human's, you need to give it a massive, detailed homework assignment. You'd show it millions of pictures and say, "Is this a golden retriever or a poodle? Is this a red sports car or a blue sedan?" The idea was that the more specific the labels, the better the robot would understand the world.

But this new paper from Johns Hopkins University suggests that the robot doesn't need that much detail. In fact, it might learn better if you give it a very simple, blurry version of the homework.

Here is the story of what they found, explained simply:

The "Blurry Map" Experiment

The researchers wanted to test if a robot could learn to see like a human with just a few broad categories instead of thousands of specific ones.

Think of the world of images like a giant library.

The Old Way (Fine-Grained): You tell the robot to sort every single book into its own specific drawer. There are 1,000 drawers, each for a specific type of book (e.g., "19th-century French poetry," "20th-century sci-fi," etc.).
The New Way (Coarse-Grained): You tell the robot to sort the books into just 8 big bins. Maybe one bin is "Animals," one is "Food," one is "Vehicles," and so on. You don't care if it knows the difference between a cat and a dog; you just want to know if it knows the difference between an animal and a car.

To do this fairly, they didn't just make up these 8 categories. They used a clever trick: they took a smart, pre-trained computer brain and asked it, "If you had to group these 1.2 million images into just 2, 4, or 8 groups based on how they look, how would you do it?" They used the computer's own "intuition" to create these broad groups.

The Big Surprise

They trained hundreds of different robot brains using these different levels of detail. Then, they checked two things:

The "Neural" Test: Did the robot's internal wiring look like the brain of a monkey or the brain scan of a human?
The "Human Feeling" Test: When the robot looked at two objects, did it think they were similar in the same way a human would? (For example, would a human say a "lion" and a "tiger" are more similar than a "lion" and a "toaster"? Did the robot agree?)

The results were shocking:

Neural Alignment: Robots trained on just 8 broad categories learned to see the world just as well as (and sometimes better than) robots trained on the full 1,000 categories. Their internal "maps" of the world looked almost identical to human and monkey brains.
Human Feelings: This is where it got even stranger. The robots trained on the coarse, 8-category task actually matched human intuition better than any other robot tested. They were better at guessing how humans group things together than the robots trained on the super-detailed 1,000-category task.

Why Does This Happen?

The paper suggests that when you force a robot to focus on the big picture (Animals vs. Vehicles), it naturally learns the most important, fundamental ways our brains organize the world.

When you force a robot to memorize 1,000 tiny details, it gets distracted by the noise. It learns to spot the tiny differences between a "red sports car" and a "blue sedan," but it might miss the forest for the trees. By simplifying the task to just a few big buckets, the robot is forced to build a strong, clear foundation that matches how humans naturally perceive things.

The "Low-Data" Bonus

The researchers also found that this "blurry map" approach is incredibly efficient.

A robot trained on the full 1.2 million images with the detailed 1,000-category task did worse than a robot trained on just 1% of the images (about 12,000 pictures) with the simple 8-category task.
It's like saying you can learn the layout of a whole city by looking at a simple sketch with 8 main districts, rather than trying to memorize every single street address in a massive atlas.

The Bottom Line

The paper concludes that to build a computer vision system that thinks like a human, you don't need to give it a PhD-level exam with thousands of specific questions. You just need to give it a very simple, coarse assignment: "Sort these things into a few big piles."

This simple feedback is enough to make the robot's brain light up and organize information in a way that is remarkably similar to our own. It turns out that sometimes, less detail leads to a clearer, more human-like understanding of the world.

Technical Summary: An Extremely Coarse Feedback Signal is Sufficient for Learning Human-Aligned Visual Representations

Problem Statement
A foundational discovery in computational neuroscience is that deep neural networks trained on visual tasks develop internal representations resembling the primate visual system. However, the field has progressively shifted toward increasingly fine-grained supervisory signals, ranging from thousand-way object classification to contrastive self-supervised objectives that distinguish individual images. A critical, unexamined question remains: how complex must an optimization objective be to yield brain-like representations? While biological development proceeds through coarse distinctions (e.g., animate vs. inanimate) before subordinate categorization, it is unclear whether the fine-grained feedback used in modern AI is necessary or even optimal for achieving alignment with human perception and neural responses.

Methodology
To systematically investigate the role of supervisory signal granularity, the authors developed a data-driven, modality-agnostic procedure to generate coarse category labels directly from the statistical structure of visual inputs, bypassing manual annotation.

Coarse-Graining Procedure: Using a pretrained source network (e.g., CLIP, AlexNet, ViT, DINOv3), the authors extracted feature representations for all ImageNet training images. They computed the principal components (PCs) of this representation space. A binary partition was created via a median split along the first principal component (PC1). Subsequent splits along PC2, PC3, etc., recursively halved the partitions, yielding $2^n$ categories (2, 4, 8, 16, 32, 64).
Experimental Design: Hundreds of neural networks were trained from scratch across diverse architectures (CNNs, ResNet-50, ConvNeXt, Vision Transformers) on these coarse classification tasks. All training variables (dataset, augmentation, hyperparameters) were held constant, isolating supervisory granularity as the sole experimental factor.
Evaluation Metrics: Representational alignment was measured using Representational Similarity Analysis (RSA). The internal representations of the trained models were compared against:
1. Macaque Electrophysiology: Multi-unit spiking responses from the THINGS Ventral-stream Spiking Dataset (TVSD).
2. Human fMRI: 7T fMRI responses from the Natural Scenes Dataset (NSD).
3. Human Behavioral Judgments: Perceptual similarity embeddings derived from ~4.7 million "odd-one-out" trials in the THINGS dataset.

Key Results

Neural Alignment: Networks trained on extremely coarse signals (as few as 2 to 8 broad categories) achieved neural alignment with both macaque and human visual cortex that matched or exceeded models trained on the standard 1,000-class ImageNet objective. This held true for both low-level (V1) and high-level (IT) visual regions.
Behavioral Alignment: Coarsely trained networks substantially outperformed all other evaluated models, including fine-grained supervised models, self-supervised models (e.g., DINOv3), and large-scale vision-language models (e.g., CLIP), in aligning with human perceptual similarity judgments. The peak performance was observed with 4–8 coarse categories.
Qualitative Differences: Visualization of representational geometries revealed that coarse supervision does not merely produce a low-resolution version of fine-grained representations. Instead, it creates a qualitatively distinct geometry with clear categorical clustering of broad domains (e.g., animals, food, vehicles) that mirrors human perception, whereas 1,000-class models exhibit more diffuse similarity spaces.
Data Efficiency: In a low-data regime, a coarse model trained on only ~1% of the ImageNet dataset achieved higher behavioral alignment than a 1,000-class model trained on the full dataset.
Generalization: The advantage of coarse supervision persisted across modern architectures (ResNet, ConvNeXt, ViT) and was not dependent on the specific source network used to derive the labels (results were consistent across CLIP, AlexNet, and pixel-based splits, though pixel-based splits failed to align, indicating the necessity of high-level semantic structure).

Significance and Claims
The paper claims that human-like visual representations emerge from remarkably coarse feedback, challenging the prevailing assumption that fine-grained optimization is required for brain alignment.

Reframing Learning Signals: The results suggest that the optimization objectives needed to learn brain-aligned representations may be far more rudimentary than previously thought. A coarse objective involving just four to eight broad domains can yield superior alignment to human perception compared to fine-grained recognition.
Principled Route to AI: Coarse categorical feedback offers a principled, annotation-free route to building AI systems with internal representations better aligned with human perception. This approach does not require direct optimization on large-scale human behavioral data.
Biological Plausibility: The findings align with developmental evidence that early visual categorization relies on coarse distinctions. The authors suggest that biological mechanisms, such as feedback from prefrontal cortex, dopaminergic reward signals, or thalamic projections, may provide such coarse supervisory signals during development.
Future Directions: The work opens a path toward hierarchical curricula where coarse feedback establishes a representational scaffold for subsequent fine-grained learning, and suggests that coarse-scale supervision could serve as a foundation for general-purpose vision models.