Geometry-Aligned LLM Fine-Tuning for Sequential Narrow-Opening Planning

1. Problem Statement

The paper addresses the challenge of rigid-body motion planning through multiple sequential narrow openings (e.g., navigating a large object through a series of doors or constrained corridors).

• Core Difficulty: Unlike single-passage planning, this problem requires long-horizon geometric reasoning. The configuration (pose) chosen to traverse an early opening strictly constrains the set of reachable configurations for subsequent openings.
• Failure Modes: Existing methods often fail because:
  • Model-based planners struggle with the high dimensionality and non-convexity of narrow passages, often failing to find globally feasible paths within time limits.
  • Learning-based planners often generate discrete waypoints that are locally valid but fail when densified into continuous motion (swept collisions) or fail to coordinate exit poses for future openings.
  • Standard LLMs lack geometric grounding and cannot guarantee collision-free continuous motion or adherence to strict physical constraints.

2. Methodology

The authors propose a Geometry-Aligned LLM Fine-Tuning Framework that generates fixed-length, machine-readable SE(2) waypoint sequences. The framework employs a bi-level training pipeline to align the LLM's output with deterministic geometric constraints.

A. Input and Output Structure

• Input: A textual prompt containing symbolic descriptions of the workspace, obstacles (axis-aligned rectangles), object geometry (vertices), start/goal poses, and explicit geometric constraints.
• Output: A structured, fixed-length sequence of waypoints ($x, y, \phi$) organized into blocks per opening.

B. Bi-Level Training Pipeline

The training process consists of two distinct stages using LoRA (Low-Rank Adaptation) adapters on a frozen base LLM (Llama-3.1-8B-Instruct):

Level 1: Failure-Driven LoRA Supervised Fine-Tuning (SFT)

• Goal: Teach the model the correct output format and common failure modes.
• Process:
  1. The model is trained on human-demonstrated waypoint sequences.
  2. Failure Feedback Loop: After each epoch, the model's output is densified into a continuous trajectory using a model-based planner and verified against four constraints:
     • (C1) Boundary (workspace limits).
     • (C2) Obstacle avoidance (static collisions).
     • (C3) Swept safety (collisions between waypoints).
     • (C4) Step-size smoothness (max translation/rotation per step).
  3. If a trajectory fails, the first failing waypoint index and the violation type are recorded.
  4. This failure note is appended to the prompt, and the model is fine-tuned to predict the correct human demonstration given this failure context. This allows the model to learn from its own geometric errors rather than just imitating data.

Level 2: Group Relative Policy Optimization (GRPO) with Geometric Verification

• Goal: Optimize for continuous-motion feasibility and long-horizon coordination, moving beyond simple imitation.
• Process:
  1. The SFT-initialized model generates a group of $G$ waypoint sequences for a given prompt.
  2. Each sequence is densified and evaluated by the geometric verifier to produce a deterministic scalar reward ($R$).
  3. Reward Function: The reward is inversely proportional to a weighted cost function ($C$) that sums violations of boundary, obstacle, and step-size constraints across the entire dense trajectory.
     $$R = \frac{1}{1 + \alpha C}$$
  4. Optimization: GRPO computes the relative advantage of each sample within the group (comparing a sample's reward to the group mean). It updates the LoRA parameters to increase the likelihood of high-reward (feasible) trajectories without requiring a separate learned reward model or human feedback.

3. Key Contributions

1. Structured Long-Horizon Reasoning: The framework leverages LLMs to generate structured waypoint blocks that explicitly coordinate exit poses from one opening to facilitate entry into the next, solving the "sequential constraint" problem.
2. Failure-Driven SFT: Introduces a training loop where structured feedback (failure index + violation type) is used to refine the model's ability to avoid specific geometric error patterns, rather than just mimicking demonstrations.
3. GRPO with Deterministic Geometric Rewards: Proposes using GRPO with a physics-based verifier to optimize the policy directly for trajectory feasibility. This aligns the generation process with full continuous-motion correctness, not just discrete waypoint validity.

4. Experimental Results

The method was evaluated on synthetic 2D environments with both In-Distribution (ID) and Out-of-Distribution (OOD) scenarios (new obstacle layouts and gap widths).

• Quantitative Performance:

  • Success Rate: The proposed method (SFT + GRPO) achieved a 92.6% success rate on ID environments and 77.3% on OOD environments.
  • Comparison: It significantly outperformed baselines:
    • Raw Llama-3.1-8B: ~8% success.
    • LoRA SFT (Imitation only): ~54.6% (ID) / ~10.4% (OOD).
    • Failure-Driven SFT (No GRPO): ~71.0% (ID) / ~31.8% (OOD).
  • Geometric Reward: The proposed method achieved the highest average geometric reward (0.94), indicating the highest quality of continuous, collision-free trajectories.
  • Failure Reduction: GRPO drastically reduced "swept collision" (C3) and step-size (C4) violations, which were the primary failure modes for the SFT-only baseline.

• Qualitative Analysis:

  • Visualizations show that the proposed method selects exit poses that are pre-oriented for the next opening, reducing the need for reorientation in tight gaps.
  • In contrast, sub-goal decomposed planners (like PRM) often produce locally feasible paths that leave insufficient space for the object to align for the subsequent opening, leading to failure upon densification.

5. Significance

• Bridging the Gap: The paper successfully bridges the gap between the high-level reasoning capabilities of LLMs and the strict geometric constraints required for robotic motion planning.
• Decoupling Perception and Planning: By using symbolic text inputs rather than visual inputs (VLMs), the method isolates the reasoning capability from perception errors, proving that LLMs can perform complex geometric reasoning when provided with exact data.
• Robustness: The use of deterministic geometric rewards in GRPO ensures that the model generalizes well to unseen environments (OOD), a common weakness in purely imitation-based learning.
• Future Impact: This approach provides a scalable framework for complex manipulation tasks in cluttered environments and sets a precedent for using reinforcement learning with deterministic verifiers to fine-tune foundation models for robotics.