🤖 AI

Probing Perceptual Constancy in Large Vision-Language Models

This study evaluates 155 Vision-Language Models across 236 experiments on color, size, and shape constancy, revealing significant performance variability and a distinct dissociation between shape constancy capabilities and those of color and size.

Original authors: Haoran Sun, Bingyang Wang, Suyang Yu, Yijiang Li, Qingying Gao, Haiyun Lyu, Lianyu Huang, Zelong Hong, Jiahui Ge, Qianli Ma, Hang He, Yifan Zhou, Lingzi Guo, Lantao Mei, Maijunxian Wang, Dezhi Luo, Ho

Published 2026-02-09

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Haoran Sun, Bingyang Wang, Suyang Yu, Yijiang Li, Qingying Gao, Haiyun Lyu, Lianyu Huang, Zelong Hong, Jiahui Ge, Qianli Ma, Hang He, Yifan Zhou, Lingzi Guo, Lantao Mei, Maijunxian Wang, Dezhi Luo, Hokin Deng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are looking at a red apple. If you walk away from it, it looks smaller. If you shine a blue flashlight on it, it looks purple. If you look at it from the side, it looks like an oval instead of a circle.

A human brain is amazing because it knows the apple is still a big, red, round apple, regardless of those changes. It doesn't get fooled by the distance, the weird light, or the angle. This superpower is called Perceptual Constancy.

This paper is like a giant report card for 155 different "AI brains" (called Vision-Language Models) to see if they have this same superpower. The researchers built a special test called ConstancyBench to check three specific skills:

1. The Three Skills Tested

Think of these skills as three different levels of a video game:

Color Constancy (The "Lighting" Level): Can the AI tell that a white wall is white, even if the room is lit by a yellow lamp or a blue neon sign?
- The Test: The AI looks at a Rubik's cube in weird lighting and has to guess the true color of the middle square.
Size Constancy (The "Distance" Level): Can the AI understand that a car far away is the same size as a car right next to it, even though the distant one looks tiny on the screen?
- The Test: The AI looks at two missiles, one far back and one up close, and has to decide if they are actually different sizes or just look different because of perspective.
Shape Constancy (The "Angle" Level): Can the AI know that a round plate is still a circle, even if it's tilted so it looks like an oval?
- The Test: The AI looks at a door that is slightly open and tilted, making it look like a trapezoid, and has to realize it's actually a rectangle.

2. The Results: Who Passed?

The researchers tested 155 different AI models, ranging from small, lightweight ones to massive, super-smart ones (like GPT-4o). Here is what they found:

The "Shape" Skill is Easy: The AI models were surprisingly good at Shape Constancy. It's like they are great at recognizing the "outline" of things. Even smaller models could figure out that a tilted rectangle is still a rectangle.
The "Color" and "Size" Skills are Hard: The models struggled much more with Color and Size. They often got tricked by the lighting or the distance. It's as if they are looking at a flat picture and forgetting that the world is 3D and has changing light.
Bigger Brains Do Better: There was a clear rule: The bigger the AI model, the better it was at these tests.
- Small models often failed, getting confused by simple tricks.
- The massive models (the "giants" of the AI world) did much better, especially at understanding size and distance. It seems that as you give the AI more "brain power" (parameters), it gets better at understanding the 3D world.

3. The Big Takeaway

The paper concludes that AI doesn't have a single, perfect "human-like" vision yet. Instead, it has a stratified (layered) vision:

It's good at simple geometry (Shape).
It's getting better at understanding the world's scale and lighting (Size and Color), but only if the model is huge.

The authors suggest that while these AI models are getting smarter, they might not be "seeing" the world the way humans do. They might just be very good at guessing based on patterns they've seen in their training data, rather than truly understanding physics.

In short: If you ask an AI to identify a shape, it's usually smart. If you ask it to figure out how big or what color something really is when the lighting or distance is tricky, it's still learning. And the bigger the AI, the less likely it is to get fooled.

Technical Summary: Probing Perceptual Constancy in Large Vision-Language Models

Problem Statement

Perceptual constancy—the ability to maintain stable perceptions of object properties (color, size, shape) despite variations in sensory input such as lighting, distance, or viewing angle—is a fundamental capability of human vision. While crucial for robust interaction in dynamic environments, it remains unclear whether current Vision-Language Models (VLMs) possess genuine perceptual constancy or merely rely on statistical correlations within their training data. As VLMs advance toward near-human performance in complex reasoning tasks, there is a need to systematically assess their cognitive capabilities regarding these fundamental perceptual mechanisms to identify limitations in their world-modeling fidelity.

Methodology

The authors introduce ConstancyBench, a large-scale benchmark derived from the CoreCognition benchmark, designed to evaluate perceptual constancy in VLMs.

Scope and Scale: The study evaluated 155 VLMs across 236 experiments. The models ranged from lightweight open-source architectures to frontier systems (e.g., GPT-4o, Gemini 1.5 Pro).
Evaluation Domains: The experiments focused on three primary dimensions of perceptual constancy:
1. Color Constancy: Assessing the ability to identify intrinsic object colors despite changes in illumination (e.g., distinguishing a green square on a Rubik's cube under different lighting).
2. Size Constancy: Evaluating the perception of stable object size despite variations in distance and retinal image size (e.g., determining if a distant missile is the same size as a closer one).
3. Shape Constancy: Testing the recognition of stable object shapes despite changes in viewing angle or perspective (e.g., identifying a rectangular door as a rectangle even when it appears as a trapezoid due to perspective).
Data Sources: The dataset comprises 253 cognitive experiments (236 used in the final evaluation) drawn from six sources: photographs/videos, classic cognitive psychology experiments, movies/animations, hand-drawn works, and AI-generated images.
Experimental Protocol: Models were tested in a zero-shot setting, generating answers and textual explanations for each prompt.
Analytical Frameworks:
- Statistical Analysis: One-way ANOVA and post-hoc Tukey HSD tests were used to compare performance across domains.
- Scaling Analysis: Regression analysis examined the relationship between model parameter count (log-scale) and performance accuracy.
- Item Response Theory (IRT): A two-parameter logistic (2PL) model was fitted to analyze the discrimination and difficulty of individual test items, characterizing the latent structure of model performance.

Key Results

1. Performance Heterogeneity and Domain Dissociation

There is significant variability in VLM performance, ranging from below 0.20 accuracy in smaller models to above 0.90 in frontier systems. Crucially, the study found a clear dissociation between the domains:

Shape Constancy: Achieved the highest mean accuracy (M = 0.723, SD = 0.170).
Color Constancy: Achieved moderate accuracy (M = 0.588, SD = 0.185).
Size Constancy: Achieved the lowest accuracy (M = 0.584, SD = 0.123).

Statistical analysis confirmed that shape constancy performance is significantly superior to both color and size constancy ( $p < 0.001$ ), while color and size constancy performance are statistically equivalent ( $p = 0.976$ ). The authors suggest shape constancy may exploit simpler geometric shortcuts, whereas color and size constancy require high-dimensional photometric representations and 3D world modeling, respectively.

2. Scaling Laws

A robust positive correlation exists between model size and perceptual stability across all domains.

Overall: Accuracy increases significantly with parameter count ( $R^2 = 0.2804$ ).
Domain Specifics: The scaling effect is strongest for size constancy ( $R^2 = 0.3080$ ), followed by shape and color constancy. This suggests that geometric and spatial reasoning benefits most directly from expanded model capacity, while photometric invariance emerges more gradually.

3. Item Response Theory (IRT) Findings

The IRT analysis revealed a structured hierarchy in task difficulty and discrimination:

Shape items showed the lowest difficulty ( $\bar{b} = -1.27$ ) and lower discrimination, indicating they are solvable by most models.
Color items exhibited the highest discrimination ( $\bar{a} = 2.25$ ), effectively separating high-ability from low-ability models, with moderate difficulty.
Size items displayed the broadest difficulty distribution, reflecting high heterogeneity in spatial reasoning demands.

Key Contributions

ConstancyBench: The establishment of the first systematic framework for probing perceptual constancy in large VLMs, covering color, size, and shape domains with 236 controlled experiments.
Empirical Characterization: The demonstration that perceptual constancy in VLMs is a stratified rather than unified ability. Shape constancy emerges robustly even in smaller models, while color and size constancy are strongly dependent on model scale and multimodal integration capacity.
Scaling Insights: The identification of a quantifiable scaling law where perceptual robustness improves with model size, with size constancy showing the steepest performance gains, suggesting a potential "emergent threshold" for complex spatial reasoning.
Cognitive Alignment: The observation that the hierarchy of constancy emergence in VLMs (shape $\rightarrow$ color/size) mirrors developmental trajectories in human perception, where geometric stability precedes higher-order contextual invariances.

Significance and Claims

The paper posits that perceptual constancy serves as a critical diagnostic lens for evaluating the world-modeling fidelity of foundation models. The authors argue that the observed stratification suggests VLMs may achieve "minimal constancy" (preserving representations through accumulated regularities) for shape, but struggle with "contextual constancy" (requiring explicit reasoning about illumination, depth, and perspective) for color and size.

The study claims that understanding these limitations is essential for advancing AI toward reliable, adaptable systems capable of operating in unpredictable real-world conditions. It raises a conceptual question regarding whether current models exhibit functional constancy (genuine invariance to physical transformations) or merely correlational constancy (a byproduct of data coverage). The authors conclude that bridging the gap between visual perception and true physical understanding in AI will require mechanistic modeling of these invariances, potentially necessitating architectural innovations beyond simple scaling, such as explicit 3D priors or causal simulation modules.