Technical Summary: Internalizing Geometric Law via Solver Residuals

Problem Statement

Large Language Models (LLMs) exhibit "precision hallucination" in domains requiring strict adherence to geometric, physical, or logical laws. While models can generate syntactically coherent and semantically plausible outputs, they frequently fail to satisfy exact metric constraints (e.g., perpendicularity, tangency, specific lengths). Existing approaches often delegate reasoning to external symbolic or numerical solvers, reducing the LLM to a translator from natural language to a solver-specific Domain Specific Language (DSL). This delegation prevents the model from internalizing spatial logic, rendering it brittle when faced with novel constraints outside the solver's predefined vocabulary.

The specific challenge addressed is Geometric Constraint Solving (GCS): translating free-form natural language descriptions into precise Python programs that output exact metric coordinates (point locations, circle radii) satisfying dozens of interacting constraints simultaneously. Unlike previous benchmarks that focus on scalar answers or theorem proofs, this task requires the generation of a fully constructed geometric configuration where all relational constraints are satisfied within a strict tolerance ($\|r\|_2^2 < 10^{-3}$).

Methodology

The PyGeoX Framework

The authors introduce PyGeoX, a programmable geometric DSL and engine designed to bridge declarative intent with differentiable optimization.

• Architecture: PyGeoX operates as a "Declarative-to-Symbolic-to-Differentiable" pipeline. It accepts high-level geometric descriptions, compiles them into symbolic constraint equations (using SymPy), and aggregates them into a single differentiable error function $E(u)$.
• Capabilities: The engine supports 35 object types and 38 relationships, including complex algebraic constraints (e.g., "perimeter of a pentagon equals the area of a circle") and inequality constraints, which are often unsupported by traditional deductive engines.
• Role in Training: PyGeoX is used strictly off-policy. It generates training data procedurally and computes per-constraint residuals during Reinforcement Learning (RL) reward evaluation. The LLM is never given access to the PyGeoX library during inference, forcing it to learn geometric laws rather than API syntax.

Reward Design: Outlier Gradient Masking and SAR

A core contribution of the paper is the identification of a failure mode in standard RL with Verifiable Rewards (RLVR) called Outlier Gradient Masking.

• The Problem: Standard global-norm rewards (e.g., $R = \exp(-\|r\|_2^2)$) aggregate all constraint residuals into a single scalar. In high-dimensional constraint spaces, a single severely violated constraint (an outlier) drives the global norm to near-zero, collapsing the reward signal for all other satisfied constraints. This "all-or-nothing" scaling nullifies the policy gradient, preventing the model from learning from partial progress.

• The Solution (SAR): The authors propose Saturating Additive Rewards (SAR). Instead of aggregating residuals globally, SAR decomposes the reward into a sum of independent, bounded kernels for each constraint:
  $$R_{SAR}(r) = \sum_{i=1}^{C} \phi(r_i)$$
  where $\phi$ is a monotonically decreasing kernel (e.g., Boltzmann $\phi(r) = e^{-r/T}$). This ensures that progress on satisfied constraints preserves a non-zero gradient signal even when other constraints are violated.

• Composite Reward: To address the "reward plateau" (where partial satisfaction yields high reward but fails to reach exact solutions) and "geometric degeneracy" (trivial solutions like collapsing all points to one coordinate), the final reward function combines:

  1. Dense Shaping (SAR): Provides continuous gradients for partial progress.
  2. Sparse Bonus: A binary reward activated only when all constraints are satisfied ($\|r\|_2^2 < 10^{-3}$).
  3. Degeneracy Penalty: Deducts reward for trivial configurations (e.g., coincident points).

Training Protocol

The framework was evaluated using Qwen3-8B as the base model.

• Data: A procedural pipeline generated ~100k validated training problems stratified into Easy, Medium, and Hard tiers.
• Approaches: The authors compared Supervised Fine-Tuning (SFT) with weighted loss (using normalized rewards) and Reinforcement Learning using Group Relative Policy Optimization (GRPO).
• Baselines: Comparisons were made against standard global-norm rewards (MSE-based), sparse binary rewards, and the proposed composite SAR+S+D.

Key Results

Performance on Benchmarks

The authors evaluated models on PyGeoX-Bench (300 stratified problems) and PyGeoX-Wild (86 out-of-distribution problems adapted from middle-school geometry benchmarks).

• SAR Superiority: The composite SAR+S+D reward significantly outperformed field-standard sparse and global-norm rewards. On the "Hard" tier, SAR+S+D achieved a solving rate (SR) of 0.41 (RL) and 0.32 (SFT), compared to 0.18 and 0.04 respectively for MSE+S+D.
• Gradient Masking Validation: Empirical analysis of 3,893 partially correct solutions showed that 60% of MSE-based rewards collapsed to near-zero, making them indistinguishable to the optimizer. In contrast, 97% of SAR rewards fell within the informative range $[0.1, 0.9]$, preserving relative gaps between solutions.
• Token Efficiency: Models trained with SAR+S+D required ~22.8% fewer tokens to solve problems compared to sparse baselines, suggesting more direct reasoning paths.
• Competitiveness: The resulting 8B model (0.41 SR on Hard) outperformed three of four evaluated frontier closed-source systems (including DeepSeek-V3.2) on the Hard tier, despite being significantly smaller.

Evidence of Internalization

The paper argues that the model internalized geometric laws rather than memorizing training data based on three lines of evidence:

1. Constructive Traces: ~90% of successful traces followed step-by-step constructive derivations (ruler-and-compass style) rather than black-box optimization calls.
2. OOD Transfer: The model maintained high performance on PyGeoX-Wild, which featured human-authored prose and unseen constraint combinations.
3. Combinatorial Infeasibility: The solution space for Hard problems exceeds $10^{17}$ configurations, dwarfing the training set, making memorization statistically impossible.

Significance and Claims

The paper claims to present the first framework that formulates end-to-end Geometric Constraint Solving as an LLM alignment task, moving beyond the "translator" paradigm. Its primary significance lies in:

1. Reward Design: Identifying and solving the "Outlier Gradient Masking" problem, demonstrating that dense, per-constraint residual signals are essential for training LLMs on multi-constraint tasks.
2. Infrastructure: Releasing PyGeoX, a differentiable geometric engine that supports inequality constraints and complex algebraic relationships, alongside PyGeoX-Bench and PyGeoX-Wild for rigorous evaluation.
3. Generalizability: The authors posit that the SAR approach is applicable to any RLVR setting where a solver returns a multi-dimensional residual vector, such as physical simulation, scene layout, or robotic manipulation.

The authors remain modest regarding the scope, noting that current results are limited to 2D static geometry and that the approach requires a base model with strong mathematical and code capabilities (as smaller models failed to converge). They do not claim to have solved general engineering design but have established a viable pathway for LLMs to internalize geometric laws in precision-critical domains.