💻 computer science

Illusion-Aware Visual Preprocessing and Anti-Illusion Prompting for Classic Illusion Understanding in Vision-Language Models

This paper presents a training-free framework for the CVPR 2026 DataCV Challenge that combines illusion-aware image preprocessing, anti-illusion prompting, and multi-vote ensembles to significantly improve Vision-Language Models' accuracy in understanding classic visual illusions by overcoming their bias toward memorized facts.

Original authors: Junli Zha, Jiahui Wang, Xinkai Lu, Jinbo Wang

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Junli Zha, Jiahui Wang, Xinkai Lu, Jinbo Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are showing a picture to a very smart, well-read robot. You point to two lines and ask, "Are these lines the same length?"

In the real world, you might be looking at a famous optical illusion (like the Müller-Lyer illusion), where arrows at the ends of the lines trick your brain into thinking one is longer than the other, even though they are identical.

Here is the problem: The robot in this paper is too smart for its own good. Instead of actually "looking" at the pixels on the screen to measure the lines, it recognizes the picture as a famous trick it has seen before in a textbook. It says, "Ah, I know this! It's the Müller-Lyer illusion. The lines are actually the same length," and answers correctly.

But here's the twist: If you slightly change the picture so the lines are actually different lengths (breaking the illusion), the robot still says they are the same. Why? Because it's relying on its memory of the trick, not its eyes to see the new reality. It's like a student who memorized the answer key but didn't learn how to do the math.

The Solution: "Tricking the Trickster"

The authors of this paper built a system to fix this "memory vs. vision" problem without needing to retrain the robot (which is like trying to rewire its brain). Instead, they used three clever tricks to force the robot to look at the picture fresh, like a human would.

1. The "Magic Eraser" (Image Preprocessing)

Instead of asking the robot to "ignore the illusion," the team physically altered the picture to make the illusion disappear.

The Analogy: Imagine trying to compare the size of two apples, but one is sitting in a dark, confusing shadow and the other is in bright light. It's hard to tell which is bigger. Instead of asking the robot to "imagine" the shadow is gone, the team literally cuts the apples out of the picture and places them side-by-side on a plain white table.
How they did it: For different types of illusions, they used specific tools:
- For color tricks: They cut out just the colored strips and put them next to each other on a gray background.
- For size tricks: They isolated the objects and "mirrored" them so they overlapped perfectly. If they were different sizes, a gap would appear; if they were the same, they would blend seamlessly.
- For straightness tricks: They drew a grid of blue lines over the image to act as a ruler.

By changing the image, they removed the "confusing context" that triggers the robot's memory. Now, the robot has to look at the actual visual evidence.

2. The "Strict Teacher" (Anti-Illusion Prompts)

Once the picture is cleaned up, the team gives the robot a very specific set of instructions (a prompt).

The Analogy: It's like a teacher telling a student, "Stop guessing based on what you read in the book. Look at the ruler I just drew on the paper. Compare the lines to the ruler and tell me what you see."
The Strategy: The prompts explicitly name the illusion (e.g., "This is a Müller-Lyer illusion") to wake the robot up, but then immediately tell it to ignore the arrows and only compare the horizontal lines. It forces the robot to switch from "recalling facts" to "doing visual comparison."

3. The "Panel of Judges" (Multi-Vote Ensemble)

Even with a clean picture and good instructions, a robot might sometimes have a "bad day" or get confused by a random glitch.

The Analogy: Imagine asking one person a question; they might get it wrong. But if you ask the same question to five different people and take the majority answer, you are much more likely to get it right.
The Strategy: The system asks the robot the same question five times and picks the answer that appears most often. This smooths out any random errors.

The Results

The team tested this system on a competition with 630 tricky images.

Without their tricks: The robot struggled, often getting stuck on its memorized knowledge.
With their tricks: The system got 90.48% of the answers correct on the official test set. On a smaller, human-verified subset, it got 98.41% correct.

They came in 2nd place in the competition, just a tiny fraction behind the winner, proving that you don't need to build a new, smarter robot to solve these problems. You just need to give the existing robot a better picture to look at and clearer instructions on how to look at it.

The Bottom Line

The paper shows that when a smart AI gets confused by optical illusions, it's usually because it's "thinking too much" based on what it already knows. The solution isn't to teach it new facts, but to clean up the image so the truth is obvious, tell it to look at the image instead of its memory, and ask it five times to be sure. It's a simple, practical way to make AI see the world more clearly.

Technical Summary: Illusion-Aware Visual Preprocessing and Anti-Illusion Prompting for Classic Illusion Understanding in Vision-Language Models

1. Problem Statement

Vision-Language Models (VLMs) exhibit a systematic bias toward visual illusions, often prioritizing memorized facts over actual visual perception. As demonstrated by the VI-Probe framework [18], VLMs frequently suffer from a "perceive-or-recall" dilemma: they correctly identify standard illusions (e.g., the Ebbinghaus illusion) by recalling the concept rather than analyzing the image, yet they fail to detect genuine physical differences when an illusion is perturbed.

This paper addresses the 5th DataCV Challenge Task 1 at CVPR 2026, which formulates classic illusion understanding as a binary Visual Question Answering (VQA) task. The goal is to develop a training-free approach that enables VLMs to provide objective, perception-grounded answers regarding seven classic illusion types (Müller-Lyer, Ebbinghaus, Poggendorff, etc.), overcoming the conflict between prior knowledge and visual evidence.

2. Methodology

The proposed solution is a three-stage, training-free pipeline that relies on visual manipulation and prompt design rather than model fine-tuning.

2.1. Illusion-Aware Image Preprocessing

Instead of asking the VLM to "see through" an illusion, the core insight is to physically modify the image to weaken or remove the illusion-inducing context. This transforms the perceptual task into a simpler visual comparison.

Discovery Process: Strategies were developed via a multi-VLM collaborative process (Stage 1: Divergent analysis by Claude-Opus, Qwen3-VL, and Gemini-3.1-Pro; Stage 2: Convergent synthesis; Stage 3: Human validation). Approximately 60% of strategies originated from model proposals, with 40% refined by humans.
Type-Specific Transformations:
- Size (Ebbinghaus/Ponzo): Isolate targets via color/intensity thresholding, then apply a mirror-blend strategy (overlapping the left and right halves). Identical sizes produce a clean overlay; different sizes reveal an edge ring.
- Color (Cornsweet/Simultaneous Contrast): Extract edge strips or crop central targets to remove surrounding gradient contexts, followed by saturation and contrast enhancement.
- Straightness (Hering/Wundt): Isolate target lines via color-channel filtering and overlay dashed reference grids to provide straightness cues against a clean background.
- Alignment (Poggendorff): Fit a linear trajectory to visible segments and render a dashed extension line across the occluding bar to visualize alignment.
- Parallelism (Café Wall): Overlay evenly spaced vertical reference lines to detect angular deviations.
- Boundary (Kanizsa): Apply contrast and sharpness enhancement to amplify real edges while keeping illusory contours faint.

2.2. Anti-Illusion Prompt Engineering

Prompts are designed to redirect the model from knowledge-based recall to qualitative visual comparison.

Mechanism Naming: Prompts explicitly name the illusion type (e.g., "This is a Müller-Lyer illusion") to activate awareness.
Feature Guidance: Instructions direct attention to specific features introduced by preprocessing (e.g., "Compare the edge ring," "Check if the red line stays parallel to the grid").
Problem Transformation: Questions are reframed to avoid triggering recall. For instance, instead of asking "Are these lines equal?", the prompt instructs the model to ignore the illusion-inducing circles and compare only the horizontal segments.

2.3. Multi-Vote Ensemble

To mitigate stochastic variance in VLM outputs, the system queries the model $N$ times per image and aggregates the results via majority voting.

Configuration: The final submission uses $N=5$ API calls.
Rationale: Experiments showed accuracy gains of +3.4 percentage points (pp) at $N=5$ compared to single queries, with diminishing returns beyond this point.

3. Key Contributions

Training-Free Framework: A complete pipeline requiring no parameter updates, relying solely on image preprocessing and prompt engineering.
Perception-Memory Decoupling: Demonstrates that physically altering the image representation is more effective than text-based guidance alone for decoupling VLMs from memorized illusion templates.
Multi-VLM Strategy Discovery: A novel methodology using multiple frontier VLMs as "visual reasoning consultants" to collaboratively discover and synthesize optimal preprocessing strategies.
Type-Specific Specialization: The development of distinct, optimized pipelines for seven different illusion categories, moving beyond generic image enhancement.

4. Experimental Results

The method was evaluated on the official 630-image test set and a human-verified 10% subset (63 samples) using Claude (claude-opus-4-6).

Overall Accuracy: Achieved 90.48% on the official test set and 98.41% on the human-verified subset.
Component Breakdown:
- Original-ACC: 98.57% (high accuracy on standard illusion images).
- Perturbed-ACC: 82.38% (accuracy on images where the illusion was modified to create genuine differences).
Ablation Study:
- Baseline (Zero-shot/Few-shot): ~62–68% accuracy.
- Image Preprocessing: Incremental improvements from denoising (+3.23pp), target highlighting (+3.12pp), and reference lines (+3.18pp) brought accuracy to 87.30%.
- Prompt + Voting: The addition of anti-illusion prompts and 5-vote ensemble provided the largest single-step gain (+11.51pp), reaching the final 98.41% on the subset.
Comparison: The full pipeline outperformed generic visual enhancement (74.51%) and various prompting baselines (CoT, Self-Consistency) by a significant margin (23.9pp over generic enhancement), confirming that type-specific transformations are critical.

The solution secured 2nd place in the challenge, trailing the 1st place by only 0.47%.

5. Significance and Claims

The paper claims that for tasks where VLM prior knowledge conflicts with visual evidence, operating on the image representation is a more promising path toward perception-grounded reasoning than operating solely on text prompts.

Decoupling Mechanism: The combination of preprocessing and prompting produces super-additive effects. Preprocessing removes the visual triggers that cause "template matching," while prompts channel the model's attention toward the relevant visual comparisons.
Practicality: The approach offers a practical, training-free strategy for improving VLM robustness in scenarios where memorized associations override active visual analysis.
Limitations: The authors acknowledge that the current approach relies on semi-automatic, manually refined per-type strategies, which may limit scalability to unseen or naturalistic scene-level illusions. Additionally, the multi-vote ensemble incurs a 5x computational cost.

The work underscores that while VLMs are susceptible to illusions due to their reliance on statistical shortcuts and prior knowledge, these failures can be mitigated by physically restructuring the input to align with the model's visual processing strengths.