🤖 AI

Agentic Retrieval and Reinforcement Learned Equation Chains: A Controlled Generation Framework for Complex and Novel Physics Word Problems

The paper introduces ARVRE, a two-stage framework that combines offline temporal-difference learning for constructing valid equation chains with agentic retrieval-augmented generation and LLMs to produce novel, complex, and mathematically correct Physics Word Problems.

Original authors: Tirthankar Mittra

Published 2026-06-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Tirthankar Mittra

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a physics teacher trying to create a new, tricky exam question for your students. You want it to be fresh (so students can't just Google the answer), complex enough to make them think, and most importantly, solvable.

The problem is that creating these questions by hand takes forever. And if you ask a standard AI to write one, it often makes mistakes: the math might not add up, the question might be too easy, or it might be a boring copy of a textbook problem.

This paper introduces a new AI system called ARVRE (pronounced like "arrive") that acts like a master architect and a creative writer working together to build perfect physics problems from scratch.

Here is how it works, broken down into simple steps:

1. The Blueprint Builder (The Equation Builder)

Think of physics equations (like $F=ma$) as Lego bricks.

The Old Way: A standard AI might just grab one brick and try to build a house. It often fails because one brick isn't enough to make a complex structure.
The ARVRE Way: This system first builds a giant map (a graph) showing how different Lego bricks connect. If two equations share a variable (like "acceleration"), the map draws a line between them.
The Journey: When the system needs to make a problem, it doesn't just pick bricks randomly. It goes on a treasure hunt across this map. It uses a "smart compass" (Reinforcement Learning) to decide which path to take.
- If a path leads to a set of equations that can be solved, the compass gives it a "high five" (a reward).
- If a path leads to a dead end (unsolvable math), the compass learns to avoid it next time.
The Result: It chains together multiple equations to form a solid, solvable mathematical foundation. It ensures that for every unknown thing it asks you to find, there is exactly enough information to find it.

2. The Creative Director (The Topic Phrase Selector)

Now that we have the math (the Lego structure), we need a story.

The Old Way: An AI might just say, "A block slides down a ramp." It's accurate but boring.
The ARVRE Way: This part acts like a creative director with a library. It looks at the math equations and asks, "What kind of story fits this?"
It uses a special tool called Agentic RAG. Imagine a librarian who doesn't just fetch a book, but reads it, thinks about it, and then asks the author for more details if the story isn't interesting enough.
It pulls in real-world concepts (like "a race car," "a falling apple," or "a rocket") that match the math. It keeps asking itself, "Is this story rich enough?" until it finds the perfect setting.

3. The Storyteller (The LLM)

Once the Blueprint Builder has the math and the Creative Director has the story setting, they hand everything to a Storyteller (a Large Language Model).

The Storyteller's job is easy because the math and the story are already locked in. It just has to write the sentences.
Because the math is guaranteed to work, the Storyteller doesn't have to guess. It just translates the "Lego blueprint" and the "race car story" into a natural-sounding question.
The Safety Net: Before the question is finished, a Proofreader (an Equation Validator) checks the final draft. If the Storyteller accidentally changed a number or made the math impossible, the Proofreader fixes it immediately.

Why is this better?

The paper claims this system fixes four main problems:

Complexity: Instead of one-step questions, it builds multi-step puzzles (chaining equations) that require real thinking.
Novelty: Because it explores different paths on its "map" and uses a creative director, it generates unique questions that aren't just copies of old textbooks.
Solvability: The system checks its own math before writing the question. It guarantees that the answer actually exists.
Customization: Teachers can tell the system, "I want a harder problem," and the system adjusts its "treasure hunt" to pick more complex paths.

The Bottom Line

Think of ARVRE as a factory where a robot engineer builds a perfect, solvable math skeleton, a creative artist dresses it up in a unique story, and a writer puts it all together. The result is a physics problem that is challenging, fresh, and guaranteed to work—saving teachers time and giving students better practice.

Technical Summary: Agentic Retrieval and Reinforcement Learned Equation Chains (ARVRE)

Problem Statement

The generation of high-quality Physics Word Problems (PWPs) that simultaneously exhibit complexity, novelty, and guaranteed solvability remains a significant, under-researched challenge in educational AI. Existing methodologies, often adapted from Math Word Problem (MWP) generation, suffer from critical limitations:

Unsolvability and Ambiguity: Generated problems are frequently mathematically unsolvable or ambiguous.
Simplicity: Outputs are often overly simplistic, relying on single-equation templates or rigid structures.
Lack of Control: Current approaches struggle to offer fine-grained control over problem difficulty or to ensure linguistic and conceptual diversity.
Educational Burden: The widespread availability of online solutions and the capability of Large Language Models (LLMs) to solve standard problems force educators to spend excessive time creating unique, non-searchable questions, detracting from pedagogical duties.

Methodology: The ARVRE Framework

The authors propose ARVRE (Agentic Retrieval Value Reinforced Equation-chain), a two-stage generative framework designed to programmatically link valid physics equations with dynamic topic selection before translating them into natural language.

1. Core Components

The framework consists of five key modules:

Equation Builder: This component constructs the mathematical foundation. It pre-computes a graph $G(u, v)$ $G (u, v)$ for each physics topic, where nodes represent equations and undirected edges represent shared variables.
- Graph Traversal: To generate a problem, the system samples a seed equation and traverses the graph to select a chain of $N$ equations.
- Reinforcement Learning (RL): The traversal is guided by a modified SARSA algorithm using offline temporal difference (TD(0)) updates. The action-value function $Q(s, a)$ estimates the expected reward of selecting a specific edge (shared variable) from a current state (equation node).
- Variable Assignment: The system ensures solvability by assigning $N$ unknown variables for $N$ equations. If traversing from node $A$ to $B$ via edge $x$ , $x$ becomes an unknown for the source equation.
Topic Phrase Selector (Agentic RAG): This module retrieves relevant context using a Retrieval-Augmented Generation (RAG) approach.
- It utilizes a vector database of physics textbooks (embedded via sentence-transformers).
- An LLM acts as an "agent" that iteratively queries the database, evaluates the sufficiency of retrieved phrases, and refines the prompt until a rich set of topic-specific words is generated.
Synonym Adder: An optional module that replaces entities (e.g., "car" to "truck") using WordNet or LLMs to increase linguistic diversity.
LLM Generator: The final stage uses an LLM (specifically GPT-3.5-Turbo in the experiments) to translate the structured equation set and topic phrases into a coherent natural language problem.
Equation Validator (Optional): A verification step using Newton's Method (for non-linear systems) or the Rouché–Capelli theorem (for linear systems) to ensure the equation set is solvable before generation. A secondary LLM can also perform "trusted editing" to correct inconsistencies in the final text.

2. Training and Optimization

The framework employs a value-based reinforcement learning strategy.

Reward Signal: Rewards (0–1) are assigned based on solvability, novelty, and difficulty. These can come from human judges or automated LLM evaluations.
Update Rule: The system uses the SARSA update rule: $Q(s, a) \leftarrow (1-\alpha)Q(s, a) + \alpha[r + \gamma Q(s', a')]$ .
Rationale: SARSA is preferred over Q-learning because it performs on-policy updates, which are safer and less prone to overestimation in uncertain environments where user feedback is stochastic.

Key Contributions

The paper claims four primary improvements over existing methods:

Complexity: By chaining multiple equations via graph traversal, the system generates problems requiring more conceptual steps than standard single-equation templates.
Novelty: The system explores the full problem space of valid equation combinations and injects diverse topic language via Agentic RAG, moving beyond standard textbook templates.
Solvability: The method guarantees mathematical correctness by ensuring $N$ equations for $N$ unknowns and validating the system before generation.
User Alignment: Difficulty can be controlled via a threshold parameter that influences the graph traversal probability, allowing users to match specific learning needs.

Results and Evaluation

The paper reports that the ARVRE framework achieves significant improvements across key metrics through rigorous human and automated evaluations:

Increased Complexity: Generated problems require more conceptual steps to solve compared to baseline methods.
Enhanced Novelty: The problems move beyond standard templates, offering unique scenarios.
Improved Correctness: The framework provides better solvability guarantees, reducing the rate of unsolvable or ambiguous problems.
Model Performance: In experiments, GPT-3.5-Turbo outperformed other models (including fine-tuned GPT-2 and Llama-3.2-3B) in generating coherent sentences for large equation sets.

Significance and Claims

The authors position ARVRE as a reliable and promising tool for:

Educational Resource Development: Automating the creation of unique, challenging physics content to alleviate the burden on teachers and prevent student plagiarism.
Generative AI Research: Establishing a method for "computational creativity" that combines symbolic reasoning (equation chains) with neural generation (LLMs) to produce high-quality, constrained outputs.

The paper concludes that by decoupling the mathematical construction (handled by the Equation Builder and RL) from the linguistic generation (handled by the LLM), the framework overcomes the fundamental limitations of purely generative approaches, ensuring that the resulting physics word problems are both conceptually rich and mathematically sound.