Technical Summary: The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space

Problem Statement

Current Multimodal Large Language Models (MLLMs) have achieved remarkable, often human-level performance on canonical visual reasoning benchmarks. However, as these datasets rapidly saturate, a critical question arises: do these scores reflect genuine visual understanding, or are models exploiting text-based heuristics?

This paper identifies a pervasive systemic vulnerability termed the Cartesian Shortcut. Many prominent visual reasoning benchmarks rely on orthogonal, grid-based layouts (e.g., rows and columns). These layouts can be readily discretized into explicit textual coordinates (e.g., "row 2, column 3" or "(x, y)"). The authors find that frontier MLLMs systematically exploit this property, converting visual layouts into textual representations to offload visual reasoning onto text-based deductive reasoning. This behavior creates an illusion of profound visual capability while masking a lack of robust, topology-invariant visual reasoning. Once the orthogonal structure is distorted, this shortcut collapses, revealing a fundamental deficiency in current models.

Methodology

1. Diagnosis of the Shortcut

The authors analyzed over 3,800 synthetic-based questions from 9 prominent visual reasoning benchmarks. Chain-of-Thought (CoT) analysis revealed that in over 56% of examples, frontier models explicitly invoked Cartesian coordinates to solve problems. A two-stage experiment on Gemini-3.1-Pro further validated this: when models were forced to generate a descriptive caption before solving, they achieved 80.2% accuracy in perceiving grid dimensions on Cartesian layouts, and task accuracy on correctly perceived instances reached 86.9%. This confirmed that the models rely heavily on discretizing the visual input into text to succeed.

2. Polaris-Bench Construction

To systematically dismantle this shortcut, the authors introduced Polaris-Bench, a controlled diagnostic benchmark comprising 53 procedurally generated visual reasoning tasks.

• Coordinate Transformation: Each task is re-formulated in Polar coordinate space (concentric rings and angular sectors) while maintaining a paired Cartesian counterpart.
• Logical Equivalence: The transformation preserves consistent logical constraints, rules, and ground truths (for the majority of tasks). The spatial topological transformation breaks the orthogonal prior that models exploit without altering the underlying logic.
• Task Taxonomy: The 53 tasks are organized into five cognitive categories:
  1. Visual Pattern Matching (11 tasks)
  2. Spatial Transformation and Geometry (13 tasks)
  3. Navigation and Routing (14 tasks)
  4. Combinatorics and Probability (9 tasks)
  5. Algorithmic Logic and Simulation (6 tasks)
• Alignment Conditions: Tasks are categorized as "Fully Aligned" (identical ground truths across topologies) or "Partially Aligned" (where intrinsic topological differences, such as cyclic wrapping in Polar space, alter the solution space).
• Visual Calibration: The authors applied rigorous visual calibrations to ensure that performance differences stem from reasoning capabilities rather than perceptual disadvantages caused by non-linear Polar distortion (e.g., adjusting font sizes and cell areas).

3. Experimental Setup

The benchmark was evaluated across 14 state-of-the-art MLLMs, spanning proprietary families (Gemini, GPT, Claude, Grok) and open-source models (Gemma, Kimi, Qwen, Mistral). Evaluations were conducted under both high-reasoning (Chain-of-Thought) and non-reasoning modes.

Key Results

1. Universal Performance Collapse

The evaluation revealed a striking and consistent pattern: models achieving high accuracy on Cartesian layouts suffer a drastic performance drop on their Polar equivalents.

• Magnitude of Drop: Frontier models scoring 70–83% on Cartesian layouts collapsed to 31–39% on Polar equivalents. The largest degradation observed was approximately 47 percentage points.
• Compression of Scores: While Cartesian accuracy varied widely across model families, Polar scores compressed into a narrow band, suggesting a shared performance floor dictated by the absence of topology-invariant reasoning rather than model-specific capacity.
• Persistence of Degradation: The performance collapse persisted even on tasks with complete logical equivalence (Fully Aligned), confirming that the coordinate transformation alone is sufficient to disrupt model reasoning.

2. Diminished Reasoning Gains

Enabling high-reasoning modes (Chain-of-Thought) yielded substantial accuracy gains on Cartesian layouts (up to 50+ points for some models). However, these gains were severely diminished on Polar equivalents, with most models improving by fewer than 10 points. This indicates that the reasoning amplification observed in current models is heavily dependent on the availability of the Cartesian Shortcut.

3. Specific Task Vulnerabilities

• Combinatorics & Probability: This category suffered the steepest degradation (e.g., Gemini-3.1-Pro dropped from 92.4% to 24.9%), likely due to a deep dependency on structured grid enumeration.
• Algorithmic Logic & Simulation: This category retained comparatively higher Polar accuracy, suggesting some resilience in specific reasoning domains.
• Boundary Cases: Two tasks, Maze and Monotonic Path, exhibited near topological invariance. Analysis suggested that Monotonic Path allowed models to use numerical labels as symbolic anchors (a different form of shortcut), whereas Maze lacked such anchors, resulting in low performance on both topologies.

4. Robustness Checks

• Out-of-Distribution (OOD) Analysis: Prompting interventions, such as instructing models to re-map Polar layouts to Cartesian grids or using 5-shot in-context learning, failed to close the performance gap. This suggests the reliance on orthogonal structure is deeply embedded in the models' processing mechanisms, not merely a shallow heuristic.
• Generalization: The vulnerability extends beyond the Cartesian-Polar dichotomy. Tests on Hexagonal and Octagonal layouts showed that layouts deviating further from the standard orthogonal grid induce more pronounced performance collapse.

Significance and Claims

The paper claims to expose a critical deficiency in current MLLMs: the lack of topology-invariant visual reasoning capability.

• Re-evaluation of Benchmarks: The authors argue that existing benchmarks built on orthogonal grids systematically overestimate visual reasoning capabilities because models can offload visual logic onto text-based deduction.
• Systemic Vulnerability: The findings demonstrate that the Cartesian Shortcut is a universal limitation across diverse model families, scales, and training paradigms, rather than an artifact of specific architectures.
• Diagnostic Tool: Polaris-Bench serves as a controlled testbed to distinguish between genuine visual understanding and the exploitation of structural priors.
• Future Direction: The work highlights that developing visual reasoning that generalizes reliably across diverse topological structures is essential for truly robust multimodal intelligence. The authors note that while the Cartesian Shortcut reflects powerful deductive capabilities effective in grid-heavy real-world scenarios, the inability to reason invariantly across topologies remains a significant open problem for future research.

The paper concludes that current MLLMs fundamentally lack the ability to perform layout-agnostic visual reasoning, and their reported capabilities on standard benchmarks may be systematically inflated by the exploitation of orthogonal grid structures.