OPTIAGENT: A Physics-Driven Agentic Framework for Automated Optical Design

This paper introduces OPTIAGENT, a physics-driven agentic framework that leverages Large Language Models enhanced with a specialized dataset, hybrid training objectives, and a physics-guided reward system to automate the design of functional lens systems, effectively bridging the gap between human expertise and automated optical engineering.

The Gist

Imagine you are trying to build a custom camera lens for a new smartphone. You need it to be sharp, capture a wide view, and let in just the right amount of light.

In the real world, this is a nightmare for computers. It's like trying to solve a 3D puzzle where every piece is made of glass, and if you move one piece by a fraction of a millimeter, the whole picture blurs. Traditionally, only highly trained human experts with years of experience could do this, and even then, it took them weeks of trial and error.

Recently, we got very smart computers called Large Language Models (LLMs) (like the AI behind ChatGPT). These AIs read millions of books and know the definitions of lenses. They can tell you, "A lens is made of glass and bends light." But if you ask them to design a working lens, they fail. They might give you a list of numbers that looks like a lens, but physically, it's impossible to build. It's like an AI writing a recipe for a cake that says "add 500 cups of flour"—theoretically it's a recipe, but in reality, it's a disaster.

Enter OPTIAGENT.

The authors of this paper created a new system called OPTIAGENT. Think of it as taking that smart AI and putting it through a rigorous "boot camp" specifically for physics. Here is how they did it, using some simple analogies:

1. The "Fill-in-the-Blanks" Training (Optical Prescription Completion)

Imagine you are teaching a student to be a master architect. Instead of just asking them to build a house from scratch, you give them a half-built house with missing bricks and ask them to figure out what goes in the empty spots.

• What they did: They created a massive dataset (called OptiDesignQA) filled with real, working lens designs. They hid some numbers (like the curve of the glass or the thickness) and forced the AI to guess the missing pieces based on the rest of the design.
• The Result: This forced the AI to stop just "guessing words" and start understanding the hidden rules of how glass pieces fit together.

2. The "Strict Coach" (Physics-Driven Rewards)

In normal AI training, the computer gets a "gold star" if it writes a sentence that sounds good. But in lens design, sounding good isn't enough; it has to work.

• The Problem: If the AI makes a mistake, a normal AI might just keep going. OPTIAGENT has a "Strict Coach" (a reward system) that checks the design at every step.
  • Level 1 (Format): Did you write the numbers in the right order? If not, zero points.
  • Level 2 (Structure): Did you accidentally make the glass pieces overlap or have negative thickness? If yes, zero points.
  • Level 3 (Physics): Does the light actually focus where it's supposed to? If not, zero points.
• The Analogy: It's like a video game where you can't move to the next level until you perfectly solve the physics puzzle. The AI learns that "looking smart" doesn't matter; "working correctly" is the only way to win.

3. The "Human-in-the-Loop" (Zemax Integration)

Even after the AI gets really good, it's still an AI. It might get the design 95% right.

• The Strategy: OPTIAGENT doesn't try to be perfect on its own. Instead, it acts as a super-fast sketch artist. It generates a "good enough" starting point in seconds.
• The Finish: It then hands this sketch to a professional, high-powered software (called Zemax) that does the final, tiny, precise adjustments.
• The Result: You get a professional-grade lens design in minutes instead of weeks.

Why is this a big deal?

Before this, if you wanted a custom lens, you needed a PhD in optics and a team of engineers. Now, with OPTIAGENT, a regular person can say, "I need a lens that is this big, sees this wide, and is this bright," and the AI will generate a working blueprint that a machine can actually build.

In summary:
The paper teaches a smart AI to stop just "talking" about lenses and start "thinking" like a physicist. By forcing it to learn the rules of light and glass through a strict reward system, they turned a text-generating robot into a capable optical engineer.

Technical Summary

1. Problem Statement

Optical lens design is a critical yet highly complex task involving the precise arrangement of refractive surfaces and materials to manipulate light. It is characterized as a highly non-convex optimization problem that traditionally relies on human heuristic expertise and iterative experimentation.

• Limitations of Traditional Methods: Automated design algorithms (e.g., evolutionary algorithms like QGSO) often struggle with discrete structural searches, producing physically invalid configurations or requiring days of computation to converge.
• Limitations of General LLMs: While Large Language Models (LLMs) possess vast theoretical optical knowledge, they fail to translate this into physically realizable designs. They lack "spatial logic," struggle with implicit parameter coupling (e.g., how curvature changes affect thickness and focal length), and cannot satisfy rigorous physical constraints (e.g., non-negative edge thickness, manufacturability). Consequently, LLM-generated designs often suffer from physical hallucinations, infeasible synthesis, and ray tracing failures.

2. Methodology: The OPTIAGENT Framework

The authors propose OPTIAGENT, a physics-driven agentic framework that reformulates optical design as a goal-oriented decision-making process using Reinforcement Learning (RL). The framework operates as a closed-loop system comprising three core modules:

A. Knowledge Injection: Optical Prescription Completion

To bridge the gap between theoretical knowledge and structural synthesis, the authors introduce an auxiliary task called Optical Prescription Completion.

• Mechanism: The model is trained on "masked" optical prescriptions where it must predict missing numerical values (radii, thicknesses, materials) based on the remaining context and design specifications.
• Goal: This forces the LLM to internalize the geometric inter-dependencies within lens systems, effectively injecting "physical intuition" into the model's weights before the main RL training.

B. Physics-Driven Policy Alignment: Optical Lexicographic Reward

The core innovation is a hierarchical reward system designed to align the LLM's policy with strict physical laws. The reward is computed in a specific order to ensure stability:

1. Format Reward ($R_{fmt}$): Ensures the output strictly adheres to the Optical Data Description Language (ODDL) syntax.
2. Structure Reward ($R_{stru}$): Validates basic physical soundness (e.g., valid object/image planes, positive air gaps, correct material assignments).
3. Paraxial Ray Tracing Reward ($R_{ray}$): Uses a differentiable paraxial ray tracing engine to verify if the generated system meets the target Effective Focal Length (EFFL) and image plane positioning.
4. RMS Reward ($R_{RMS}$): Calculates the Root Mean Square spot radius to optimize image quality. This is only activated if the lower-level constraints (format, structure, and basic ray tracing) are satisfied.

• Total Reward: $R_{lex} = R_{fmt} \cdot R_{stru} \cdot (R_{ray} + \delta_{pass} R_{RMS})$. This lexicographic structure prevents the model from optimizing for image quality on physically broken structures.

C. Optimization Strategy

• Training Algorithm: The framework employs DrGRPO (Group Relative Policy Optimization Done Right) to optimize the policy.
• Inference Pipeline: The agent generates an initial robust structure ($L_0$). This structure is then fed into Zemax (a commercial optical design software) for local optimization to achieve commercial-grade precision. The authors restrict Zemax usage to the inference stage to keep training efficient and focus the agent on global structural synthesis.

3. Key Contributions

1. Pioneering Agentic Framework: OPTIAGENT is the first framework to successfully apply LLMs to optical lens design by reformulating it as a physics-driven RL problem, enabling non-experts to generate functional lens systems.
2. OptiDesignQA Dataset: The authors curated the first dedicated dataset for optical design LLMs, containing:
   • 711 whole design tasks and 124 prescription completion tasks for training.
   • 80 whole design tasks for testing.
   • Data sourced from authoritative textbooks and novel configurations generated by state-of-the-art global optimization algorithms.
3. Physics-Driven Policy Alignment: The introduction of the Optical Lexicographic Reward and the Optical Prescription Completion task effectively bridges the "spatial logic" gap, allowing LLMs to generate designs that strictly satisfy physical constraints.

4. Experimental Results

The method was benchmarked against traditional optimization algorithms and state-of-the-art LLMs (including ChatGPT-5.2, Claude Sonnet 4.5, and Qwen3-235B).

• Success Rate (SR): OPTIAGENT achieved a 90.1% success rate (physically viable designs), significantly outperforming baselines (e.g., Qwen3-235B at 77.8%, ChatGPT-5.2 at 72.0%).
• Precision: It achieved a marginal 1.0% relative error in Effective Focal Length (EFFL), compared to errors ranging from 35% to 54% in baselines.
• Image Quality: The initial RMS spot radius generated by OPTIAGENT was 672.09 $\mu m$, outperforming competitors by an order of magnitude (e.g., Qwen3-235B at 3288.30 $\mu m$).
• Ablation Studies:
  • Removing the hierarchical reward structure (e.g., combining Ray Tracing and RMS naively) led to training divergence.
  • The Optical Prescription Completion task was crucial; without it, performance dropped significantly.
  • RL vs. SFT: Direct Reinforcement Learning outperformed Supervised Fine-Tuning (SFT), proving that policy alignment is superior to simple pattern imitation for complex physical reasoning.

5. Significance

• Democratization of Optical Design: OPTIAGENT allows users without formal optical training to generate high-fidelity lens systems via natural language, removing the barrier of complex technical parameter tuning.
• Bridging AI and Physics: The work demonstrates that LLMs can be effectively aligned with rigorous physical constraints through hierarchical rewards and differentiable simulators, moving beyond simple text generation to physics-aware synthesis.
• Efficiency: By generating high-quality initial structures, OPTIAGENT drastically reduces the time and computational resources required for traditional optical design, which often takes days to converge.
• Future Impact: This framework paves the way for autonomous optical engineering systems and establishes a new paradigm for solving complex, non-convex physical problems using agentic AI.