🤖 AI

AlphaOPT: Formulating Optimization Programs with Self-Improving LLM Experience Library

AlphaOPT is a self-improving framework that enables large language models to learn and refine optimization modeling knowledge through a continual cycle of extracting solver-verified insights from failures and evolving their applicability, thereby achieving superior generalization and performance on complex optimization tasks without requiring costly retraining or gold-standard program labels.

Original authors: Minwei Kong, Ao Qu, Xiaotong Guo, Wenbin Ouyang, Chonghe Jiang, Han Zheng, Yining Ma, Dingyi Zhuang, Yuhan Tang, Junyi Li, Shenhao Wang, Haris Koutsopoulos, Hai Wang, Cathy Wu, Jinhua Zhao

Published 2026-06-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Minwei Kong, Ao Qu, Xiaotong Guo, Wenbin Ouyang, Chonghe Jiang, Han Zheng, Yining Ma, Dingyi Zhuang, Yuhan Tang, Junyi Li, Shenhao Wang, Haris Koutsopoulos, Hai Wang, Cathy Wu, Jinhua Zhao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: Teaching a Robot to Do Math

Imagine you have a brilliant but inexperienced apprentice (a Large Language Model, or LLM) who is very good at writing stories and chatting. You want to teach this apprentice to solve complex optimization problems—like figuring out the most efficient way to deliver packages, schedule factory machines, or manage a budget.

To do this, the apprentice must translate a messy, natural-language description (e.g., "We need to ship goods from three warehouses to five stores without exceeding truck limits") into a precise mathematical formula and then write computer code to solve it.

The Catch:

Prompting is brittle: If you just ask the apprentice nicely, they often get the math wrong because they don't "know" the rules of operations research.
Retraining is expensive: You could try to retrain the apprentice on thousands of examples, but most of those examples only give the final answer, not the step-by-step thinking. It's like giving a student the answer key but not the textbook; they memorize the answers but can't solve new problems.

The Solution: AlphaOPT (The "Self-Improving Library")

The authors created AlphaOPT, which is like giving the apprentice a growing, self-correcting notebook (an experience library) instead of just a one-time lecture.

Think of it as a Master Chef and a Junior Cook scenario:

The Junior Cook (The LLM) tries to cook a complex dish (solve a problem) based on a recipe description.
The Master Chef (The Solver) tastes the dish. If it's burnt or salty, the Master Chef says, "Wrong! It needs less salt and more heat."
The Notebook (The Library): Instead of just fixing the dish, the Junior Cook writes down a specific lesson in a notebook: "When the recipe mentions 'high heat' but the pan is small, reduce the flame to prevent burning."

How AlphaOPT Works: The Two-Phase Cycle

The system runs in a continuous loop with two main phases:

Phase 1: Library Learning (The "Trial and Error" Phase)

The Attempt: The LLM tries to solve a problem. If it fails, it doesn't just give up. It uses the "Master Chef" (a mathematical solver) to check its work.
The Extraction: Once the LLM finally figures out the right answer, the system looks at what it got wrong before and what fixed it.
The Entry: It writes a structured note into the library. This note isn't just a random sentence; it's a 4-part recipe:
1. Category: What kind of problem is this? (e.g., "Transportation").
2. Condition: When does this rule apply? (e.g., "Only when there are fixed costs for opening a warehouse").
3. Explanation: Why does this rule exist?
4. Example: A concrete snippet of code or math showing the fix.

Phase 2: Library Evolution (The "Refinement" Phase)

This is the magic part. The notebook isn't static; it gets smarter over time.

The Diagnosis: The system looks at all the problems it has tried. It asks: "Did this rule help? Did it confuse us? Did we miss a problem where this rule should have applied?"
The Fix:
- If a rule was too broad (e.g., "Always use Big-M constraints"), and it caused errors in some cases, the system tightens the rule: "Use Big-M only if there is a binary choice involved."
- If a rule was too narrow (e.g., "Only for trucks"), and it could have helped with "trains" too, the system broadens the rule.
The Result: The library evolves from a list of specific tips into a set of general, reliable principles that work across many different types of problems.

Why This is Different (The "Secret Sauce")

Most other AI systems try to learn by:

Memorizing patterns (Fine-tuning): Like a student cramming for a test. They fail if the test questions look slightly different.
Guessing based on text similarity (Prompting): Like a librarian finding a book because the title sounds similar, even if the content is wrong.

AlphaOPT is different because:

It checks the math: It doesn't just trust the AI's word. It runs the code through a solver. If the math doesn't add up, the lesson isn't saved.
It understands "When": It doesn't just store a rule; it stores the conditions for using the rule. It knows when to apply a specific math trick and when to avoid it.
It gets better with more data, without retraining: As you feed it more problems, the library grows and refines itself. The AI doesn't need to be retrained from scratch; it just updates its notebook.

The Results: What Happened?

The researchers tested AlphaOPT on several difficult datasets:

It got smarter over time: As they added more training problems (from 100 to 300), the success rate went up steadily (from 65% to 72%).
It handled the unexpected: When tested on problems it had never seen before (Out-of-Distribution), it crushed the competition. While other methods dropped significantly in performance, AlphaOPT stayed strong.
It beat the best: It outperformed the strongest existing methods by a significant margin (around 8-9% better).

The Bottom Line

AlphaOPT is a system that lets an AI learn to do complex math and coding by making mistakes, checking the answers with a calculator, and writing down structured lessons that get refined over time. It's like giving the AI a mentor that helps it turn every failure into a permanent, reusable skill, allowing it to solve new, unseen problems with high accuracy.

Technical Summary: AlphaOPT

Problem Statement

Optimization modeling is critical for decision-making in industries such as finance, manufacturing, and logistics, yet automating the translation of natural-language problem descriptions into precise mathematical formulations and executable solver code remains a significant challenge. Existing Large Language Model (LLM) approaches face two primary limitations:

Prompt-based systems are brittle, relying on fixed templates that are sensitive to wording changes and domain shifts, lacking long-term memory or sustained knowledge injection.
Fine-tuning approaches require costly retraining on new data and often depend on high-quality, expert-annotated datasets containing intermediate reasoning or gold-standard programs, which are scarce in the operations research (OR) community. Most available datasets provide only final solutions, limiting the model's ability to learn the structural reasoning required for modeling decisions.

Furthermore, existing experience learning frameworks (e.g., Reflexion, ExpeL) designed for general reasoning tasks are ill-suited for optimization. They often store experiences as unstructured text or edits without explicit applicability semantics, lack rigorous solver-based validation, and organize knowledge as flat lists rather than structured hierarchies, leading to negative transfer when applied to structurally constrained optimization problems.

Methodology: AlphaOPT

The authors propose AlphaOPT, a self-improving experience library framework that enables LLMs to learn optimization modeling knowledge from limited supervision (specifically, answer-only feedback without gold-standard programs). AlphaOPT operates without updating model parameters, instead maintaining and evolving an external library of structured insights through a continual two-phase cycle:

1. Library Learning Phase

This phase focuses on extracting and organizing structured insights from failed attempts.

Insight Extraction: When a generated solver program fails to achieve the optimal objective, the system performs solver-guided self-exploration. It iteratively proposes programs, reuses prior failures as context, and verifies correctness via the solver. A successful program serves as a proxy for the gold standard to extract insights.
Structured Representation: Each insight is formulated as a four-tuple: (Taxonomy, Condition, Explanation, Example).
- Taxonomy: Hierarchical labels (Level-1 and Level-2) for indexing (e.g., Domain Modeling $\to$ Network Flow $\to$ Flow Conservation).
- Condition: Explicit prerequisites and triggering signals describing when the insight applies.
- Explanation: The underlying principle of the modeling rule.
- Example: Concrete demonstrations (mathematical expressions or code snippets).
Verification: New insights undergo strict retrieval and execution verification to ensure they resolve the original failure and are retrievable before being stored.
Storage & Merging: Insights are stored in a dynamically updating hierarchical taxonomy dictionary. An online merging mechanism prevents redundancy by checking new insights against existing ones.

2. Library Evolution Phase

This phase refines the applicability conditions of stored insights to prevent over-generalization or under-generalization.

Diagnosis: The system analyzes interactions between tasks and insights, categorizing them into three sets for each insight $i$ $i$ :
- $S^+_i$ : Positive tasks where the insight was applicable and contributed to success.
- $S^-_i$ : Negative tasks where the insight was misleading and degraded performance.
- $S^u_i$ : Unretrieved tasks where the insight would have been beneficial but was not retrieved.
Refinement: Based on aggregate evidence across these sets, the LLM refines the Condition field of the insight. Strategies include adding preconditions, introducing keyword anchors, or explicitly excluding scenarios (inapplicability clauses) that caused negative transfers.
Optimization Objective: The refinement process aims to maximize a performance score $p_i$ that balances keeping positive cases, correcting negative cases, and recovering unretrieved cases, effectively adjusting the insight's retrieval boundary in the problem space.

Key Contributions

First Experience Learning Framework for OR Modeling: AlphaOPT is the first framework to adapt experience learning specifically for natural-language optimization modeling, representing solver-verified knowledge as condition-aware insights that are continually refined.
Structured, Solver-Verified Knowledge: Unlike prior methods storing unstructured text, AlphaOPT uses a hierarchical taxonomy and solver verification to ensure the structural validity and transferability of insights.
Aggregate-Driven Refinement: The framework introduces a mechanism to refine insight applicability conditions using cross-task evidence, ensuring insights are neither too narrow nor too general.
Performance with Limited Supervision: The system achieves strong results using only answer-level supervision, avoiding the need for expensive gold-standard program annotations.

Experimental Results

The authors evaluated AlphaOPT on multiple benchmarks, including NLP4LP, NL4OPT, IndustryOR, and MAMO, as well as out-of-distribution (OOD) datasets LogiOR and OptiBench.

State-of-the-Art Performance: AlphaOPT achieved the highest accuracy on most benchmarks. On the test splits, it reached 87.7% on NLP4LP and 60.0% on IndustryOR.
Out-of-Distribution Generalization: AlphaOPT demonstrated superior robustness compared to fine-tuned models (e.g., ORLM, LLMOPT) and prompt-based baselines. On the OOD LogiOR dataset, AlphaOPT achieved 56.0%, significantly outperforming ORLM (19.6%) and retrained LLMOPT (27.2%). On OptiBench, it reached 93.5%.
Continual Improvement: The system showed steady performance growth as training data increased from 100 to 300 items (Micro-avg rising from 65% to 72%), while the library size converged, indicating efficient knowledge accumulation without unbounded redundancy.
Ablation Studies: Removing key components (Taxonomy, Applicability Conditions, Refinement, or Examples) resulted in significant performance drops, confirming that the structured retrieval and refinement mechanisms are critical for success.

Significance and Claims

The paper claims that AlphaOPT demonstrates that structured experience learning grounded in solver feedback provides a practical alternative to retraining for complex reasoning tasks requiring precise formulation and execution.

Key claims include:

Interpretability and Auditability: The structured library offers explicit, human-inspectable knowledge transfer, exposing characteristic failure patterns in domain modeling, formulation, and solver syntax.
Generalization: The framework captures transferable modeling principles rather than dataset-specific patterns, enabling strong generalization to unseen problem types and domains.
Scalability: The convergence of the library size suggests that the system scales efficiently, with token costs growing sub-linearly as the library stabilizes.
Broader Applicability: While focused on optimization, the authors note that the framework generalizes to other knowledge-intensive tasks, such as algorithm design and general-purpose code generation, suggesting it could serve as a general paradigm for self-improving agents.

The authors conclude that this approach paves the way for more challenging settings, such as efficient program formulation and large-scale industrial optimization, by providing a mechanism for LLMs to accumulate and refine reusable modeling principles over time.