Technical Summary: FormalAnalyticGeo

Problem Statement

While Multimodal Large Language Models (MLLMs) have advanced significantly in mathematical reasoning, analytic geometry remains a largely underexplored domain. Existing benchmarks predominantly focus on elementary plane or solid geometry, and the few large-scale resources available for conic sections (e.g., Conic10K) are purely text-based, lacking the visual diagrams essential for multimodal training.

The paper identifies three specific gaps hindering the automated generation of multimodal analytic geometry problems:

1. Representation Gap: Existing formalisms (e.g., Inter-GPS, FormalGeo) target Euclidean constructions and cannot express coordinate systems or conic curves. Conversely, text-only encodings (e.g., Assertional Logic in Conic10K) lack the construction primitives required for rendering diagrams.
2. Rendering Gap: Analytic geometry diagrams require algebraically defined curves within coordinate systems. Traditional plotting libraries require pre-computed positions and cannot handle constraint-driven layouts where positions are under-determined by text alone.
3. Verification Gap: Current generation approaches are typically single-pass. Errors in problem formulation, diagram rendering, or answer extraction propagate undetected, resulting in low-quality datasets without human annotation.

Methodology: FormalAnalyticGeo Framework

The authors propose FormalAnalyticGeo, a scalable, neural-symbolic framework that generates multimodal analytic geometry problems without human annotation. The system operates as a closed-loop pipeline comprising four specialized LLM components and a deterministic rendering engine, connected by a formal intermediate language.

1. Condition Description Language (CDL)

To bridge the gap between natural language and visual rendering, the authors introduce CDL, a declarative formal language.

• Structure: CDL programs consist of semicolon-separated statements categorized into declarations (defining objects like lines, circles, ellipses), equations (algebraic definitions), coordinates, derived objects (e.g., foci, intersections), predicates (geometric relationships), and metric constraints.
• Function: Every valid CDL statement maps directly to a visual element or a differentiable constraint, enabling automatic syntactic validation and completeness checking.

2. SDF-Based Rendering Engine

The framework employs a Signed Distance Field (SDF) engine to compile CDL programs into geometric diagrams.

• Unified Representation: Unlike traditional tools that separate constraint solving from rendering, the SDF engine treats both as a single differentiable process. Geometric elements (conics, axes, asymptotes) are represented as scalar functions $F: \mathbb{R}^2 \to \mathbb{R}$.
• Constraint Solving: Under-determined positions (e.g., a point on a curve satisfying a distance constraint) are resolved via gradient descent over geometric constraints.
• Exact Mapping: The rendering process produces an exact coordinate-to-pixel mapping, which is critical for the subsequent ground-truth extraction.

3. The Generation Pipeline

The framework executes a sequential workflow with feedback-driven retries:

• Generator: Produces diverse natural language problems and conic equations. It utilizes a "reverse path" grounded in a seed library of verified properties (from Conic10K) to ensure geometric consistency, alongside a "forward path" for free composition. It enforces an information partition where some conditions are text-only and others are visual-only.
• Formalizer: Translates the natural language problem into a CDL program. It uses grammar prompting tailored to specific curve types to reduce hallucinations and is immediately parsed for syntax errors.
• SDF Engine: Renders the diagram from the CDL.
• Measurer: Extracts ground-truth answers via physical information isolation. It operates in two stages:
  1. Task Extractor: Reads only the problem text to generate a structured measurement instruction (no equations).
  2. Visual Measurer: Reads only the instruction and the rendered diagram, using computer vision tools (e.g., pixel_to_coord, intersect_line_curve) to measure values. This prevents the model from bypassing visual measurement by solving the equations analytically.

4. Quality Verifier and Closed-Loop Control

A dedicated Quality Verifier enforces data quality at three gate positions:

1. Gate 1 (Problem): Verifies solvability via symbolic solution.
2. Gate 2 (CDL): Validates syntax and ensures all geometric objects in the text are encoded in the CDL.
3. Gate 3 (Answer): Checks the plausibility of the visual measurement against a text-only symbolic solution.
   If a gate fails, the verifier returns structured feedback to the upstream component, triggering an automatic retry. This loop eliminates the need for human annotation.

Key Contributions

1. Closed-Loop Generation Framework: A system that decomposes multimodal data generation into specialized stages connected by a formal language, utilizing feedback-driven quality gates to eliminate human annotation.
2. CDL (Condition Description Language): A formal intermediate representation that bridges natural language descriptions and geometric rendering, enabling automatic validation.
3. SDF-Based Rendering Engine: A differentiable engine that unifies constraint solving and diagram rendering, allowing for exact analytic curves and precise coordinate-to-pixel mappings.
4. AnalyticGeo7K Dataset: A large-scale dataset of over 7,000 verified multimodal problems, each containing aligned text, diagrams, formal annotations, and visually extracted ground truths.
5. Comprehensive Empirical Analysis: Experiments on 8 state-of-the-art MLLMs demonstrating the difficulty of the domain and the necessity of visual input.

Results and Evaluation

• Dataset Quality: The generated AnalyticGeo7K dataset contains 7,043 verified problems. The Measurer achieves a median ground-truth relative error of 0.70%, with 82.3% of answers falling within 5% of the exact symbolic solution.
• Framework Reliability: The three quality gates achieve a final pass rate of $\ge$ 99%. Gate 1 (Problem) passes 99.1% on the first try; Gate 2 (CDL) passes 98.2%; Gate 3 (Answer) passes 95.1%, with the remaining failures triggering retries or discarding the sample.
• Model Performance: On the AnalyticGeo7K benchmark, the best-performing model (Gemini 3 Flash with image input) achieves 77.6% accuracy. Performance drops significantly in text-only mode (best text-only result: 42.0%), highlighting the critical role of visual diagrams in analytic geometry reasoning.
• Ablation Studies:
  • Removing the Quality Verifier causes a dramatic drop in accuracy (within-5% accuracy falls from 82.3% to 45.5%).
  • Removing CV Tools from the Measurer (forcing visual estimation without tools) increases mean error from 2.84% to 40.5%.
  • Removing the Seed Library increases the retry rate and reduces geometric coherence, confirming the value of structural priors.

Significance and Claims

The paper claims that FormalAnalyticGeo addresses the scarcity of annotated multimodal analytic geometry data by providing a fully automatic, annotation-free generation framework. By unifying formal language (CDL), differentiable rendering (SDF), and multi-agent verification, the framework produces high-quality, verified datasets that enable the training and evaluation of MLLMs on complex spatial and algebraic reasoning tasks. The authors assert that their work fills a critical gap in multimodal mathematical reasoning benchmarks, moving beyond Euclidean plane geometry to the more challenging domain of conic sections and coordinate geometry. The framework and the resulting AnalyticGeo7K dataset are released to facilitate further research in this area.