🤖 AI

GRASP: Gated Regression-Aware Skill Proposer for Self-Improving LLM Agents

GRASP is a self-improvement framework for LLM agents that iteratively updates a bounded skill library through gated acceptance based on a hard regression budget, significantly boosting performance on structured clinical and non-clinical benchmarks while preventing the degradation of previously learned behaviors.

Original authors: Johannes Moll, Jean-Philippe Corbeil, Jiazhen Pan, Martin Hadamitzky, Daniel Rueckert, Lisa Adams, Keno Bressem

Published 2026-05-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Johannes Moll, Jean-Philippe Corbeil, Jiazhen Pan, Martin Hadamitzky, Daniel Rueckert, Lisa Adams, Keno Bressem

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a very smart but slightly clumsy robot assistant how to navigate a complex building, like a hospital. The robot has a natural talent for talking, but when it comes to following strict rules (like "check the patient's ID before opening the door" or "don't give two pills at once"), it keeps making the same mistakes.

The paper introduces a new way to teach this robot, called GRASP. Think of GRASP not as a teacher who just gives the robot a long, never-ending list of notes, but as a strict editor who manages a small, high-quality "cheat sheet."

Here is how it works, broken down into simple concepts:

1. The Problem: The "Note-Taking" Trap

Previous methods tried to improve robots by letting them write down notes after every mistake.

The Analogy: Imagine a student who, after every wrong answer on a test, writes a new rule on a piece of paper and sticks it to their forehead.
The Issue: Over time, the student's forehead is covered in so many notes that they can't read the important ones. Worse, a new note might fix one specific mistake but accidentally break a rule they were already doing correctly. The robot gets confused and starts failing at things it used to do well. This is called "regression."

2. The Solution: The "Gated" Cheat Sheet

GRASP changes the game. Instead of just adding notes, it treats the robot's knowledge as a small, bounded library of skills (like a deck of cards).

The Editor: When the robot fails, a "skill writer" (another AI) suggests a new rule or a change to an existing one.
The Gatekeeper (The "Gated" Part): This is the most important part. Before the new rule is allowed into the cheat sheet, it must pass a strict test.
- The system takes the new rule and runs it against a "practice exam" containing both the mistakes the robot used to make and the things it used to get right.
- The Rule: The new rule is only accepted if it fixes more old mistakes than it creates new ones. If it fixes one problem but breaks two things that were working fine, the Gatekeeper says, "Nope, rejected."

3. The "Regression Budget"

GRASP has a hard limit on how much damage is allowed.

The Analogy: Imagine you are renovating a house. You can knock down a wall to fix a leak, but you are only allowed to break one other thing in the process. If you break two windows to fix the leak, the renovation is cancelled.
The Result: This ensures the robot never gets worse at what it already knows. It only gets better.

4. What Happened in the Experiments?

The researchers tested this on five different "brains" (AI models) using two very strict medical computer environments (where the robot has to look up patient records and manage medicine orders).

The Results:
- Without GRASP, the robots were like novices, getting about 40% of tasks right.
- With GRASP, the robots became experts, getting nearly 90% right.
- The Key Finding: The paper proves that the magic wasn't in the robot writing the rules. If you let the robot write rules without the Gatekeeper's strict test, it performed just as badly as if it had no rules at all. The improvement came entirely from the filtering process (the Gatekeeper) that stopped bad rules from entering the library.

5. The "Master Teacher" Effect

The paper found something fascinating about transferring knowledge between robots.

The Analogy: Imagine a master chef (a very strong AI) writes a cookbook. If you give that cookbook to a junior chef (a weaker AI), the junior chef cooks much better than they could on their own.
The Catch: If the junior chef tries to write a cookbook for the master chef, the master chef actually gets worse.
Why? The master chef's cookbook contains specific, high-level tricks for that kitchen that the junior chef can follow. But the junior chef's notes are often too simple or slightly wrong for the master chef's complex needs. GRASP is the only method that successfully preserves this "Master Chef" knowledge when moving it to a weaker robot.

6. Where Does It Work?

GRASP is like a specialized tool. It works incredibly well in environments where:

There are clear, repeatable rules (like a database or a medical record system).
You can easily check if a step was done correctly (e.g., "Did the medicine get dispensed? Yes/No").

It does not work well in open-ended situations where the rules are vague or the "right" answer is hard to define (like chatting casually or navigating a chaotic, unpredictable world). In those cases, the robot's performance stays flat.

Summary

GRASP is a method for teaching AI agents by curating their knowledge rather than just accumulating it. It acts like a strict editor who only lets new instructions into the agent's brain if they are proven to fix problems without breaking anything else. This turns a clumsy robot into a reliable, procedural expert, specifically in structured environments like hospitals or databases.

Technical Summary: GRASP (Gated Regression-Aware Skill Proposer)

Problem Statement

Large Language Model (LLM) agents operating in structured environments (e.g., electronic health records, databases, web interfaces) frequently fail due to operational violations of environmental rules rather than conversational misunderstandings. Reliability in these contexts depends on procedural knowledge specific to the environment, not general language capability.

Prior self-improvement methods typically accumulate natural-language guidance (reflections, rules, or skills) monotonically during inference. A critical flaw in these approaches is the lack of validation against existing behavior: a new note or rule that corrects one failure trajectory may silently regress performance on previously correct trajectories. Over time, this unbounded accumulation dilutes task-relevant content, degrades the agent's context window, and leads to catastrophic forgetting or oscillating performance.

Methodology: GRASP

The authors introduce GRASP (Gated Regression-Aware Skill Proposer), a framework that treats agent self-improvement as a sequence of validated edits to a bounded library of behavioral instructions (skills), without updating model parameters.

Core Components

Structured Skills: Skills are Markdown documents with YAML frontmatter containing a trigger condition, a behavioral rule, an optional verification step, and a contrastive example. They are injected into the agent's context at inference.
Failure-Driven Proposal:
- Failed trajectories are grouped by mechanism-specific failure modes (e.g., date_filter_omitted rather than generic wrong_answer).
- A skill-writing model proposes edits (ADD, MODIFY, or REMOVE) for the most frequent failure modes, utilizing active skill summaries and labels of other failure modes to discourage regressions.
- The library is capacity-bounded; adding a skill requires removing one unless capacity is available.
Regression-Aware Selection (The Gate):
- Before acceptance, every candidate edit is evaluated on a balanced held-out probe consisting of previously failing and previously passing examples from the development split.
- The system computes a baseline ( $F_0, R_0$ ) by re-running the current library on the probe.
- A candidate library ( $S_c$ $S_{c}$ ) is accepted only if:
  1. It produces a net improvement: $(F(c) - F_0) - (R(c) - R_0) > 0$ .
  2. It respects a hard regression budget: $R(c) \le R_0$ (no new regressions beyond the existing baseline).
- If the best candidate causes regressions, a contrastive revision step attempts to narrow the trigger or add guard clauses.

Training Protocol

GRASP operates in batches over a development split. After each batch, the skill writer generates candidates, which are scored on the probe. The best valid candidate is applied to the library. The process repeats for multiple epochs. No parameter updates occur; only the skill library evolves.

Key Contributions

Gated Editing vs. Monotonic Accumulation: GRASP shifts the paradigm from unbounded accumulation of guidance to bounded, validated editing. This prevents the "silent regression" problem where fixing one error breaks another.
Validation of the Mechanism, Not Just the Content: Ablation studies demonstrate that the performance gains are driven by the validation gate and comparative proposal generation, not merely the act of writing skills. Without the gate, skill-writing performs no better than using no skills.
Cross-Model and Cross-Benchmark Transfer:
- Asymmetric Transfer: Libraries written by stronger models (e.g., GPT-5.4) significantly improve weaker executors on out-of-distribution (OOD) tasks, often exceeding what the weaker models learn for themselves. The reverse (weaker to stronger) does not hold.
- Environment-Specific Knowledge: This asymmetry suggests the libraries encode environment-specific procedural knowledge rather than model-specific patterns.
- Generalization: The mechanism improves performance in non-clinical environments (ALFWorld, WebShop) where tasks have verifiable structure, but fails to improve performance where the action space is open-ended (OS Interaction).
Efficiency and Auditability: GRASP maintains a small, versioned library (average ~5 skills, ~5.6k tokens) that is significantly smaller than unbounded memory baselines (34k–53k tokens), reducing inference costs. The structured format allows for human audit and reversible edits.

Experimental Results

The authors evaluated GRASP across five base models (gpt-oss-120b, DeepSeek V4 Flash, Gemini 3.1 Flash Lite, GPT-4.1, GPT-5.4) on two FHIR-based clinical benchmarks (MedAgentBench, MedAgentBench-v2) and four non-clinical AgentBench environments.

MedAgentBench Performance: GRASP substantially outperformed five baselines (including memory-based and rule-extraction methods).
- On gpt-oss-120b, GRASP lifted accuracy from 40.6% (no skills) to 88.8%, a gain of 48.2 points over the baseline and 21.0 points over the strongest competitor (Evo-MedAgent).
- Similar improvements were observed across all models, with gains ranging from 17.2 to 40.3 points over their respective baselines.
- On the OOD split, GRASP achieved a 25.0-point lead over the best baseline on gpt-oss-120b.
Ablation Findings:
- Removing the acceptance gate caused performance to drop to the level of the no-skills baseline (40.1% vs 40.6%).
- Removing the regression budget reduced performance by 7.0 points.
- "Fixes-only" selection (ignoring passing examples) reduced performance by 8.6 points and increased variance.
Transfer Results:
- Applying a GPT-5.4-trained library to gpt-oss-120b improved OOD accuracy from 56.3% to 77.8% (+21.5 points).
- Cross-benchmark transfer worked between MedAgentBench variants but degraded performance on FHIR-AgentBench, indicating the skills are tied to specific interface conventions rather than general clinical competence.

Significance and Claims

The paper claims that treating agent self-improvement as gated editing of a bounded library offers a robust path toward iterative refinement in environments where procedural reliability is critical.

Procedural Reliability: The method addresses the specific failure mode of prior verbal-feedback methods where monotonic accumulation degrades context.
Distillation without Parameters: The asymmetric cross-model transfer suggests that stronger models can distill operational facts about an environment into a library that weaker models can follow, even if they cannot generate those facts themselves.
Practical Deployment: The bounded, auditable nature of the skill library makes it suitable for deployment in high-stakes domains (like clinical settings) where agents must be inspectable and correctable without full retraining.
Limitations: The authors note that the validation step is the dominant computational cost. Furthermore, the skills are policies tuned to benchmark scoring criteria (exact-match against ground truth) rather than medical expertise, requiring clinician review for real-world deployment. The method is also limited by the fixed action budgets of certain benchmarks, which no method successfully optimized.

In summary, GRASP demonstrates that validated editing is the primary driver of improvement in structured agent environments, outperforming methods that rely solely on skill generation or unbounded memory accumulation.