🤖 machine learning

Remember, Don't Re-read: Stateful ReAct Agents for Token-Efficient Autonomous Experimentation

This paper introduces a stateful ReAct agent architecture using LangGraph that replaces the token-inefficient, stateless autoresearch pattern with a persistent state mechanism, achieving significant token reduction (up to 90%) across hyperparameter tuning and code optimization tasks while maintaining comparable performance.

Original authors: Faramarz Jabbarvaziri

Published 2026-06-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Faramarz Jabbarvaziri

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a scientist trying to find the perfect recipe for a cake. Every time you bake a batch, you taste it, write down what happened, and then decide what to change for the next batch.

The Old Way (Stateless Agent): The "Amnesiac Chef"
In the traditional method described in the paper, the AI chef has a terrible memory. Every single time it bakes a new batch, it doesn't just look at the last cake. It pulls out a giant notebook containing the history of every single cake it has ever baked, from the very first one to the present.

It reads the entire notebook again, word for word, just to remember what it did before.

The Problem: If you bake 10 cakes, the notebook is small. But if you bake 100 cakes, the notebook is huge. The chef spends most of its time and energy just reading the old notes instead of thinking about the new cake. The paper calls this an O(n²) problem: as the number of experiments grows, the work grows exponentially. It's like trying to find a specific page in a book by reading the entire book from page 1 every single time you need a new page.

The New Way (Stateful Agent): The "Organized Librarian"
The authors of this paper built a smarter system using a tool called LangGraph. Instead of a chef with amnesia, imagine an AI with a personal librarian.

The Librarian Keeps the History: The librarian (the "State") holds the entire history of every cake baked, every tweak made, and every result observed. The AI doesn't need to carry this heavy book around.
The Summary Note: When the AI needs to decide what to do next, the librarian doesn't hand over the whole library. Instead, the librarian writes a tiny, 3-sentence sticky note: "You tried 25 batches. The best one was 1.9x faster. Here are the last 5 attempts you made. Don't forget to avoid the mistake you made on batch #7."
The Result: The AI only reads this short note. It doesn't matter if the AI has baked 10 cakes or 1,000 cakes; the note is always the same short length. The work required stays constant (O(1)).

What They Tested
The researchers tested this "Librarian" system against the "Amnesiac Chef" on two specific tasks:

Tuning a Radio (Hyperparameter Tuning): They tried to find the perfect settings for a machine learning model.
- The Result: The Librarian system used 90% fewer words (tokens) to get the same result. It was like the chef stopped reading the whole history book and just looked at a cheat sheet.
Fixing a Slow Computer Program (Code Optimization): They tried to make a slow piece of code run faster by rewriting it.
- The Result: Even though the "notes" here were longer (because they had to include the actual code), the Librarian still used half as many words as the Amnesiac Chef. The Amnesiac Chef kept re-reading the entire history of code changes, while the Librarian just showed the current best version and the last few attempts.

The Big Takeaway
The paper isn't about making the AI "smarter" at finding the best cake or code. Both methods found solutions of similar quality.

The breakthrough is efficiency.

The Old Way: As you run more experiments, the cost (in computer time and money) explodes because the AI has to re-read everything every time.
The New Way: The cost stays flat. You can run hundreds of experiments without the AI getting bogged down by its own history.

In Short:
The paper shows that by giving an AI a persistent "memory bank" (State) and a way to summarize that memory (ReAct Agent), you stop it from wasting energy re-reading its own diary. It allows the AI to run long, complex experiments without getting overwhelmed by the sheer size of its own history.

Technical Summary: Stateful ReAct Agents for Token-Efficient Autonomous Experimentation

Problem Statement

The "autoresearch" pattern enables autonomous experimentation by having a Large Language Model (LLM) iteratively modify code to optimize a target metric. While effective, the standard implementation is stateless: at every iteration, the agent must reconstruct the entire experimental context by re-reading the full history of results and code modifications accumulated in a research directive file (e.g., program.md).

This design incurs a structural inefficiency:

Token Cost: The prompt size grows linearly ( $O(n)$ ) with the number of iterations, leading to a total token cost of $O(n^2)$ .
Redundancy: The agent re-transmits information it has already processed in previous steps.
Scalability: As experiments accumulate, the prompt approaches context window limits, necessitating truncation or summarization, which risks losing critical experimental nuance.

The paper posits that because LLMs cannot maintain transient state across interactions (Huang et al. [10]), external state management is an architectural necessity to break this quadratic cost curve.

Methodology

The authors reformulate the autoresearch pattern as a stateful ReAct agent using LangGraph. Instead of monolithic prompts containing full history, the agent interacts with the ML system via tool calls, while experimental history, reasoning traces, and convergence metrics persist in a typed persistent state graph.

Architecture

The agent operates as a state graph with three primary nodes:

REASON: Invokes the LLM (Claude Haiku 4.5) with the current state context. The model generates reasoning traces and tool calls.
TOOLS: Executes tool calls deterministically (e.g., running experiments, querying results) and returns observations.
CHECK: Evaluates convergence criteria (target metric threshold, iteration budget, or LLM conclusion) without LLM involvement.

Conditional edges route the flow: if the LLM produces tool calls, control passes to TOOLS; otherwise, it proceeds to CHECK. If not converged, the loop returns to REASON.

State Management

The core innovation is the AgentState (a typed dictionary) which persists across iterations. It includes:

messages: A sliding window of the last 20 messages to bound input cost.
experiment_history: A list of all past results.
current_best: The best result found so far.
train_py_content: The current code state.

Crucially, the agent uses a helper class (ExperimentState or CodeState) to generate a compact summary for the LLM at each step. This summary includes:

Current iteration count and best aggregate metrics.
The last 5 experiments (or recent results).
Strategy notes and failed approaches.

This ensures the LLM receives only $O(1)$ input tokens per iteration, regardless of the total number of experiments run. The full history remains accessible in the state but is excluded from the prompt unless explicitly queried via tools.

Tooling

The agent interacts with the ML system through specific tools:

Hyperparameter Tuning: get_experiment_history and run_experiment.
Code Optimization: get_current_train_py, modify_train_py (with guardrails to preserve evaluation logic), submit_training_job, and query_results.

Key Contributions

Architectural Reformulation: Transforming the stateless autoresearch pattern into a stateful ReAct agent using LangGraph, decoupling the agent's working memory from its prompt context.
Token Efficiency: Demonstrating a structural reduction in token cost from $O(n^2)$ (stateless) to $O(n)$ (stateful) total, with $O(1)$ per-iteration cost.
Implementation Blueprint: Providing a detailed, reproducible architecture (including state schemas, graph logic, and tool definitions) for practitioners to implement stateful agents.

Experimental Results

The authors evaluated the stateful agent against a stateless baseline on two benchmark tasks using 3 random seeds.

Task 1: Hyperparameter Tuning (Small Observations)

Setup: 15 iterations optimizing XGBoost on the UCI Covertype dataset. Observations were small (~200 tokens: JSON config + metrics).
Results:
- Token Reduction: The stateful agent consumed 90% fewer tokens (2,492 vs. 24,465), a 9.8× reduction.
- Performance: Both agents achieved comparable F1 scores (0.779 vs. 0.764), with differences attributed to seed variance rather than architectural superiority.
- Cost Curve: The stateless input cost grew linearly from ~700 to ~1,900 tokens per iteration; the stateful cost remained flat at ~100 tokens.

Task 2: Code Performance Optimization (Large Observations)

Setup: 40 iterations optimizing a Python data-processing function. Observations were large (~1,500–3,000 tokens: full source code + benchmarks).
Results:
- Token Reduction: The stateful agent consumed 52% fewer tokens (627K vs. 1,275K), a 2.0× reduction.
- Performance: Both agents achieved comparable speedups (~1.9× over baseline).
- Cost Curve: While the stateful agent still incurred a fixed cost to send the "current best code" (~1,500 tokens) at every step, the total cost plateaued. In contrast, the stateless cost continued to climb linearly, accumulating over 80,000 tokens of history by iteration 40.

Comparative Analysis

The token reduction ratio is inversely proportional to the observation size relative to the state summary.

Small Observations: The fixed-size summary is dramatically smaller than the accumulated history, yielding high reduction ratios (9.8×).
Large Observations: The stateful agent must still transmit the current code artifact, limiting the ratio (2.0×). However, the paper notes that as iterations increase (e.g., to 100 or 200), the $O(n^2)$ vs. $O(n)$ divergence will cause the ratio to widen significantly (projected 4× to 7×).

Significance and Claims

The paper claims that the primary contribution is structural efficiency, not necessarily improved optimization quality. The stateful agent achieves comparable optimization quality to the stateless baseline while drastically reducing token consumption.

Scalability: The $O(1)$ per-iteration cost enables arbitrarily long experiment sequences without prompt truncation or loss of decision-making capacity.
Economic Impact: While the savings are modest for short runs, they become material for large-scale research (hundreds of experiments daily), where the quadratic cost of stateless execution becomes a dominant bottleneck.
Future Outlook: The authors suggest that as experiment horizons lengthen, the quadratic token cost of stateless execution will become the primary constraint, making stateful architectures essential for sustainable autonomous experimentation.

The authors remain modest regarding optimization quality, explicitly stating that observed differences in metrics are within noise and should not be interpreted as quality advantages of the stateful approach. The focus remains strictly on the efficiency gains of the architectural shift.