Perceptual similarity judgments reflect one's own discrimination capacities

This study demonstrates that subjective perceptual similarity judgments are not arbitrary but are normatively grounded in and predictive of an individual's own objective perceptual discrimination capacities.

The Gist

The Big Question: Why Do We Think Things Look Alike?

Imagine you are looking at two faces. You might say, "Wow, those two look very similar!" But why? Is it because they have the same nose? The same skin tone? Or maybe just because you are tired and your brain is being lazy?

For a long time, scientists have been puzzled by this. They call it an "ill-posed problem." It's like asking, "How similar are a banana and a boat?" The answer depends entirely on what you are looking at. If you focus on color, they are similar (yellow). If you focus on shape, they are totally different. There is no single "correct" answer, which makes it hard to study how our brains work.

The Authors' Big Idea:
The researchers (Ali Moharramipour and his team) had a clever hypothesis. They thought: "Maybe the reason you think two things look similar is simply because your eyes and brain struggle to tell them apart."

In other words, your subjective feeling of "they look alike" is actually a report card on your own visual superpowers. If you can't tell the difference between two faces, you will naturally say they are similar.

The Experiment: The "Face Gym" and the "Face Guessing Game"

To test this, they put 12 people through two different challenges using 30 different computer-generated faces.

1. The "Face Guessing Game" (Subjective Similarity)

First, the participants played a game. They saw one "Target" face at the top and four "Candidate" faces at the bottom. They had to rank the candidates from "Most Similar" to "Least Similar" to the target.

• The Catch: There were no right or wrong answers. They just had to go with their gut feeling.
• The Goal: To map out how each person felt about the faces.

2. The "Face Gym" (Discrimination Capacity)

Next, they tested the participants' actual visual muscle. They took two faces and created a smooth video transition (a "morph") between them, like a slider going from Face A to Face B.

• The Challenge: They showed the participant three faces: two identical ones and one that was slightly different (just a tiny step away on the slider).
• The Task: The participant had to spot the "odd one out."
• The Twist: The researchers made the task harder and harder until the participant could barely tell the difference. This measured their JND (Just Noticeable Difference).
  • Analogy: Imagine trying to hear a whisper in a noisy room. If you have great hearing, you can hear a very quiet whisper (small JND). If your hearing is average, you need the whisper to be louder (large JND).

The "Aha!" Moment

The researchers compared the two sets of data:

1. How similar the person said the faces were.
2. How hard it was for that person to actually tell the faces apart.

The Result:
They found a perfect match!

• If a pair of faces was hard for a person to distinguish in the "Face Gym," that same person rated them as very similar in the "Face Guessing Game."
• If a pair of faces was easy to distinguish, they rated them as very different.

The Best Part: This wasn't just a general rule for everyone. It was personal.

• Person A might struggle to tell Face X and Face Y apart, so they think they are twins.
• Person B might have great eyes for those specific faces, so they think they are strangers.
• The study proved that your personal opinion of similarity is a direct reflection of your personal visual limits.

Why Does This Matter? (The Metaphors)

1. The "Quasi-Objective" Truth
Usually, we think "similarity" is just a feeling in your head, totally made up. But this study says: No, it's grounded in reality. Your brain is doing a smart calculation. It's saying, "I can't see the difference, so for me, they are the same." It turns a subjective feeling into something measurable, like a ruler for your eyes.

2. The "AI vs. Human" Difference
The authors make a fascinating point about Artificial Intelligence. AI can be trained to say, "These two faces look similar" based on data. But an AI doesn't see. It doesn't have eyes that get tired or blurry.

• Human Similarity: "These look alike because my brain can't find the difference." (It's personal and biological).
• AI Similarity: "These look alike because the data says so." (It's statistical and cold).
  This suggests that for AI to truly be "conscious" like us, it would need to have its own "visual limits" to base its opinions on.

3. The "Warping" of Reality
Imagine your mind is a map of the world.

• If you have great vision, the map is stretched out. Two towns that are close together look far apart on your map because you can see the details between them.
• If you have blurry vision, the map is squished. Those same two towns look right next to each other because you can't see the space between them.
  Your "similarity judgment" is just you reading your own, unique, slightly warped map.

The Bottom Line

We often think our opinions are just random guesses. But this study shows that when we say, "Those two look the same," we are actually giving a very accurate report on our own sensory abilities. We don't just see the world; we see the limits of our own vision, and we call that "similarity."

Technical Summary

1. Problem Statement

Perceptual similarity is fundamental to human learning and generalization, yet it is often considered an "ill-posed" problem. The core issue is the lack of an objective ground truth for similarity judgments: it is unclear which features or combinations of features an observer should prioritize when judging two stimuli as similar (e.g., judging faces based on skin color vs. gender features). Consequently, similarity judgments are viewed as purely subjective and potentially arbitrary, lacking a normative basis.

The authors hypothesize that subjective similarity judgments are not arbitrary but are normative and rational, grounded in an individual's specific perceptual discrimination capacities. Specifically, they propose that stimuli judged as "similar" are those that are objectively harder for that specific individual to discriminate near the perceptual threshold.

2. Methodology

Participants and Stimuli:

• Participants: 12 healthy adults.
• Stimuli: 30 synthetic faces generated using the Basel Face Model (BFM), ensuring a diverse range of shape and texture variations.

Experimental Tasks:
The study employed two distinct tasks to measure subjective similarity and objective discrimination capacity:

1. Subjective Similarity Judgment Task (Supra-threshold):

   • Participants viewed a target face and four candidate faces.
   • They ranked the candidates from most to least similar to the target.
   • This was performed over 330 trials (using an InfoTuple method to maximize information gain) across two sessions.
   • Data Processing: Ranking data were converted into a $30 \times 30$ dissimilarity matrix. The authors used 5-dimensional Metric Multidimensional Scaling (MDS) to reconstruct the matrix, reducing noise and estimating the Euclidean distance between faces in a psychological space. A machine-learning approach (minimizing a loss function based on ranking probabilities) was also pre-registered and used for validation, showing slightly higher consistency.

2. Near-Threshold Discrimination Task (Objective):

   • For a selected subset of 24 face pairs (18 "controversial" pairs with high inter-subject variance in similarity ratings, and 6 "less-controversial" pairs), participants performed a psychophysical discrimination task.
   • Procedure: A "1-up-2-down" staircase method was used. Participants viewed three faces (two identical, one different) and had to identify the odd one out.
   • Stimulus Generation: 1,000 morph steps were generated between each face pair. The difficulty was titrated to find the Just Noticeable Difference (JND)—the minimum morph steps required for 71% correct performance.
   • Metric: The study quantified discrimination capacity as the number of JNDs (#JNDs) between two faces (calculated as $1000 / \text{JND steps}$). A higher #JND indicates greater perceptual distance (higher discrimination capacity).

Hypotheses:

• Hypothesis 1: There is a positive correlation between an individual's perceptual discrimination capacity (#JNDs) and their subjective perceptual dissimilarity ratings. (i.e., harder-to-distinguish pairs are rated as more similar).
• Hypothesis 2: This association is individual-specific. An individual's similarity judgments are better predicted by their own discrimination capacity than by the discrimination capacities of other participants.

Statistical Analysis:

• Spearman correlations were computed for each participant between their dissimilarity matrix and their #JNDs.
• Group-level significance was assessed via bootstrapping (100,000 iterations) to generate 95% confidence intervals for the mean z-values.
• Hypothesis 2 Test: A non-parametric permutation test was used. For each participant, their dissimilarity ratings were correlated with the #JNDs of other participants (permuted data) to create a null distribution. The actual correlation with their own data was tested against this null.

3. Key Results

Data Quality:

• Consistency: Within-participant correlations between the two sessions of similarity judgments were significantly higher than between-participant correlations ($z > 2.3$ for most), confirming that similarity judgments are stable and unique to individuals.
• Robustness: The staircase procedures for #JNDs converged well, with reversal ratios indicating reliable threshold estimation.

Hypothesis 1 (General Association):

• A strong, positive association was found between perceptual discrimination capacity and subjective dissimilarity.
• Statistics: Group-mean z-value = 2.63 (95% CI [2.18, 3.10]).
• Significance: $p < 0.00001$ (t-test), Bayes Factor (BF) > 10,000.
• Interpretation: Face pairs that were harder to discriminate (lower #JNDs) were consistently rated as more similar by the participants.

Hypothesis 2 (Individual Specificity):

• The relationship between discrimination capacity and similarity judgments was significantly specific to the individual.
• Statistics: Group-mean z-value = 1.19 (95% CI [0.77, 1.59]).
• Significance: $p = 0.00010$ (t-test), BF = 306.
• Interpretation: An individual's subjective similarity ratings were significantly better predicted by their own psychophysical thresholds than by the thresholds of other people.

Additional Findings:

• The association was stronger for "controversial" face pairs (high variance in group ratings) than for "less-controversial" pairs, likely due to measurement noise obscuring the signal in low-variance data.
• Reducing the dimensionality of the similarity space from 5D to 2D MDS weakened the results, suggesting that 5D captures the necessary complexity of the perceptual space.

4. Key Contributions

1. Normative Grounding of Subjectivity: The study provides empirical evidence that subjective similarity judgments are not arbitrary but are grounded in an individual's "quasi-objective" perceptual limits. It resolves the "ill-posed" nature of similarity by proposing that the observer's own discrimination capacity serves as the ground truth.
2. Individual-Specific Perceptual Spaces: It demonstrates that the psychological space in which similarity is represented is unique to each individual, shaped by their specific sensory resolution.
3. Methodological Framework: The paper introduces a rigorous framework combining free-response similarity ranking with near-threshold psychophysics (staircase methods) to link high-level cognitive judgments with low-level sensory capacities.

5. Significance and Implications

• Metacognition: The findings suggest that similarity judgments are a form of metacognition. They reflect an implicit readout of one's own sensory capabilities. Instability in similarity judgments may stem from inaccurate self-assessment of these capacities.
• Neural Mechanisms: The results imply that higher cortical areas (e.g., prefrontal cortex), known for metacognition, likely integrate sensory signals from visual cortices to form similarity judgments.
• Artificial Intelligence & Consciousness: The authors argue that current Large Language Models (LLMs) that mimic human similarity ratings lack "consciousness" in this specific sense. While LLMs can predict human ratings based on statistical world knowledge, they do not possess perceptual capacities to ground those judgments. Therefore, their similarity metrics lack the "subjective experience" component tied to an individual's sensory limits.
• Generalization: The findings support Roger Shepard's universal law of generalization, suggesting that the "distance" in psychological space is directly modulated by an individual's perceptual resolution. Higher resolution leads to steeper decay in generalization.

In conclusion, the paper establishes that perceptual similarity is a reflection of perceptual capacity, transforming similarity judgments from subjective whims into measurable, normative indicators of an individual's sensory system.