Synthesizing Multimodal Geometry Datasets from Scratch and Enabling Visual Alignment via Plotting Code

This paper introduces GeoCode, a synthetically generated multimodal geometry dataset that ensures consistency through code-based diagram rendering and leverages plotting code as an explicit alignment objective to significantly enhance the geometric reasoning capabilities of vision-language models.

The Gist

Imagine you are trying to teach a robot how to solve complex geometry problems. You show it a picture of a triangle with some lines and angles, and you ask, "What is the length of this side?"

Current robots (AI models) are good at reading the text of the question, but they are terrible at "seeing" the picture. They often guess the answer based on the words they've seen before, rather than actually understanding the shapes in the image. It's like a student who memorizes the answers to a math test without ever learning how to do the math.

This paper, "GeoCode," introduces a brilliant new way to teach these robots by building a massive, perfect practice library from scratch and forcing them to learn the "blueprint" of the picture, not just the words.

Here is the breakdown of their approach using simple analogies:

1. The Problem: The "Blind Architect"

Right now, AI models are like blind architects. If you give them a blueprint (the math logic) and a photo of a building (the diagram), they can often guess the height of the building just by reading the text description. But if you cover the text and only show them the photo, they get lost. They can't translate the visual lines and angles into the logical rules needed to solve the problem.

2. The Solution: Building a "Perfect Factory"

Instead of trying to fix the robots by showing them more random pictures from the internet, the authors built a factory that creates geometry problems from scratch.

Think of this factory as a three-step assembly line:

• Step 1: The Logic Skeleton (The Seed)
  First, they use a super-smart logic engine (like a digital mathematician) to build the rules of a geometry problem. They decide, "Okay, we need a triangle where two lines are perpendicular, and a circle touches one side." They haven't drawn it yet; they just have the logical rules. This ensures the problem is mathematically possible.
• Step 2: The Builder (The Coder)
  Next, they use an AI "Builder" to turn those abstract rules into a computer program (specifically, plotting code). This code is like a set of precise instructions: "Draw a point at (0,0), draw a line to (5,5), draw a circle with radius 3."
  Because this code generates the image, the picture is guaranteed to match the rules perfectly. There are no errors, no "impossible" shapes, and no contradictions.
• Step 3: The Editor (The Debiaser)
  Finally, they act as a strict editor. They take the text description of the problem and delete any clues that are already obvious in the picture.
  • Bad Example: "In the picture, you can see a right angle at A. Also, the text says angle A is 90 degrees." (The robot just reads the text and ignores the picture).
  • Good Example: "In the picture, you see a right angle at A. Find the length of side B." (The robot must look at the picture to know it's a right angle).

3. The Secret Sauce: "Teaching the Blueprint"

This is the most creative part of the paper. Usually, when we teach a robot, we show it a picture and ask for the answer.

The authors say: "No, first, tell us the blueprint."

They train the robots to look at the geometry diagram and write the plotting code that would recreate that image.

• The Analogy: Imagine you show a student a finished Lego castle. Instead of asking, "How many bricks are in the tower?" they ask, "Write the instructions on how to build this castle from scratch."
• Why this works: To write the instructions (the code), the robot must understand exactly where every point is, which lines connect to which, and what the angles are. It forces the robot to "see" the structure of the image deeply, rather than just guessing the answer.

4. The Results: From "Guessers" to "Thinkers"

They tested this new method on existing geometry benchmarks (standard tests for AI).

• Before: The robots struggled with complex, multi-step problems.
• After: The robots trained on this "GeoCode" dataset got significantly better. They didn't just memorize answers; they learned how to look at a diagram, understand the hidden structure, and solve the problem logically.

Summary

The authors didn't just give the AI more homework; they built a perfect, self-checking textbook where every picture matches the math perfectly. Then, they changed the test: instead of just asking for the answer, they made the AI write the code to draw the picture.

This forced the AI to stop guessing and start truly "seeing" the geometry, turning a blind architect into a skilled engineer.

Technical Summary

1. Problem Statement

Multimodal geometry reasoning requires models to jointly understand visual diagrams and perform structured symbolic inference. Current Vision-Language Models (VLMs) struggle with complex geometric constructions due to two primary bottlenecks:

1. Data Scarcity and Quality: Existing datasets often rely on manual design or limited symbolic sampling, resulting in low structural diversity, shallow reasoning chains, and a lack of strict cross-modal consistency (between text, image, and logic).
2. Weak Visual-Symbolic Alignment: Most methods supervise models only on final answers or natural language reasoning traces. This leaves the critical step of recovering geometric structure from diagrams as an implicit, latent process. Models often rely on linguistic correlations or dataset biases rather than explicitly reconstructing geometric relations from visual input.

The authors argue that the dominant bottleneck is not logical inference (which formal solvers can handle), but the conversion of visual layouts into faithful symbolic representations.

2. Methodology

The paper proposes a three-stage pipeline to synthesize complex multimodal geometry problems from scratch, ensuring strict consistency across symbolic structure, text, reasoning, and images.

A. The Generation Pipeline

The synthesis process is factorized into three modular, verifiable stages:

1. Seed Generation (Symbolic Structure):

   • Instead of random sampling, the system uses AlphaGeometry (via the GenesisGeo framework) to construct symbolic geometric seeds.
   • It samples predicate sets (e.g., perpendicularity, parallelism, collinearity) and uses forward symbolic reasoning to derive dependency graphs.
   • A "Target Finder" identifies non-trivial deductive chains, filtering out tautologies or degenerate cases.
   • Output: Abstract problem skeletons containing symbolic premises, proof steps, and target conclusions, devoid of numerical values.

2. Grounded Instantiation (Numerical & Textual):

   • Instructor Module: An LLM assigns concrete numerical values (e.g., Pythagorean triples) to the symbolic seeds, generates natural language problem statements, and produces Chain-of-Thought (CoT) reasoning traces.
   • Coder Module: A separate LLM generates meta-code (Python) to compute the coordinates of all geometric entities based only on the problem statement (not the solution). This ensures the visual construction is grounded in the text, not the answer.
   • Two-Stage Verification:
     • Semantic Validation: Checks logical coherence between the problem, reasoning, and answer.
     • Geometric Validation: Executes the meta-code to generate coordinates and numerically verifies all constraints (lengths, angles, parallelism) and the final answer. Inconsistent samples are discarded.

3. Visualization and Debiasing:

   • Rendering: The verified coordinates are rendered into clean, textbook-style diagrams using OpenCV.
   • Textual Debiasing: The problem statement is rewritten to remove information that can be directly inferred from the diagram (e.g., "points A, B, C are collinear" if the image clearly shows it). This forces the model to rely on visual perception rather than redundant text cues.

B. Plotting Code as Explicit Alignment

A core innovation is the use of Plotting Code as an explicit supervision target.

• Instead of just predicting an answer, models are trained to predict a structured JSON-like plotting code that defines points, segments, circles, and annotations.
• This transforms visual understanding into a structured prediction task. To generate the code, the model must explicitly reconstruct the geometric scene (dependencies, spatial relations) from the image.
• This acts as a strong intermediate supervision signal, bridging the gap between visual perception and symbolic reasoning.

3. Key Contributions

1. GeoCode Dataset: Construction of a high-quality, 18K multimodal geometry dataset. It includes diagrams, solutions, and plotting code for every instance. The dataset features significantly higher structural complexity and reasoning difficulty than existing benchmarks (e.g., Geometry3K, GeoQA).
2. Synthesis Pipeline: A novel end-to-end pipeline that decouples symbolic seed construction, grounded instantiation, and code-based visualization, ensuring mathematical correctness through multi-stage validation.
3. Code-Based Alignment Strategy: The introduction of plotting code prediction as an explicit alignment objective. This forces models to learn geometric structure recovery directly from images, rather than relying on implicit latent representations.
4. Empirical Validation: Demonstration that models trained on GeoCode achieve consistent improvements across multiple public benchmarks, including out-of-distribution (OOD) settings.

4. Experimental Results

• Dataset Quality: GeoCode exhibits higher structural complexity (average ~10 predicates per problem) and reasoning depth compared to standard datasets. Manual inspection of a random subset confirmed 100% correctness.
• Performance Gains:
  • Models (Qwen2.5-VL and Qwen3-VL) trained on GeoCode showed significant improvements on public benchmarks.
  • Notable gains were observed on OlympiadBench (+11.5% for Qwen3-VL), a challenging benchmark requiring multi-step reasoning.
  • Improvements were also seen on Geometry3K, MathVerse, and GeoQA, indicating the learned representations are transferable and not overfitted to the synthetic distribution.
• Ablation Studies:
  • Code vs. Caption: Supervising models with plotting code yielded substantially better results than supervising with natural language captions. This confirms that explicit structural grounding is superior to weakly structured text descriptions.
  • Structural Recovery: There is a strong positive correlation between the model's ability to correctly predict the plotting code (segment-level F1) and its final solving accuracy. This proves the model is learning to reconstruct geometric structures, not just memorizing answer formats.
  • Debiasing: Textual debiasing provided additional gains only when combined with code supervision, highlighting that removing text shortcuts is most effective when the model is forced to perform explicit structural reconstruction.

5. Significance

This work addresses a fundamental limitation in multimodal reasoning: the lack of explicit, verifiable alignment between visual diagrams and symbolic logic.

• Methodological Shift: It moves beyond "answer-only" supervision to "structure-aware" supervision, treating geometric diagrams as complete structural representations that should be explicitly recovered.
• Scalability: The pipeline demonstrates that high-quality, mathematically rigorous multimodal data can be synthesized at scale without human annotation, provided strict verification mechanisms are in place.
• Future Impact: The approach offers a blueprint for improving VLMs in other domains requiring precise structural understanding (e.g., circuit diagrams, chemical structures) and suggests that code-based intermediate representations are a powerful tool for aligning vision and reasoning.

In summary, GeoCode and the associated plotting code alignment strategy provide a robust solution to the visual-symbolic gap in geometry reasoning, proving that explicit structural supervision leads to more robust and generalizable multimodal intelligence.