🤖 machine learning

ISM:Self-Improving Strategy Memory for Continual Mathematical Reasoning

The paper proposes Intelligent Schema Memory (ISM), a self-evolving memory system that enhances the continual mathematical reasoning of frozen large language models by maintaining a compact, verified bank of strategy schemas, thereby outperforming existing baselines with significantly fewer stored strategies under strict episodic resets.

Original authors: Prakhar Dixit, Tim Oates

Published 2026-07-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Prakhar Dixit, Tim Oates

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant math tutor who is incredibly smart but has a very strange rule: they cannot learn from their mistakes. Every time they solve a problem, their brain is wiped clean. They remember nothing from the previous hour, the previous day, or even the problem they just solved five minutes ago. If you ask them a new question, they start from scratch, as if they've never seen math before.

This is the situation with many powerful AI models (called "frozen LLMs"). They are great at isolated tasks, but if you give them a long stream of different math problems, they keep stumbling because they can't build on what they learned yesterday.

The paper introduces a solution called ISM (Intelligent Schema Memory). Think of ISM not as a new brain for the tutor, but as a super-smart, self-cleaning filing cabinet that sits right next to the tutor.

Here is how it works, broken down into simple concepts:

1. The "Recipe Book" vs. The "Index Card"

Most systems that try to help AI remember things just dump every single past example into a giant pile. It's like giving the tutor a library of 10,000 books and saying, "Find the right one!" This gets messy and slow.

ISM is different. Instead of storing whole problems, it stores "Strategy Schemas."

The Content (The Recipe): This is the actual advice, like "When you see a geometry problem with circles, try drawing a radius."
The Feature Hook (The Index Card): This is a label that helps the system find the right recipe quickly.

The genius of ISM is that it keeps the "Recipe" (the advice) stable, but it constantly updates the "Index Card" (the label) based on how well that recipe worked. This means the system gets better at finding the right advice without changing the advice itself.

2. The Self-Cleaning Janitor

The filing cabinet has a built-in janitor (the Memory Controller) that runs a self-improvement loop. It doesn't just let the cabinet fill up; it actively manages it using seven specific tools:

The Auditor: Checks if a recipe is still useful. If it's been used and failed, it flags it.
The Merger: If two recipes are basically the same, it combines them into one to save space.
The Pruner: If a recipe hasn't been used in a while or keeps failing, it throws it in the trash.
The Reinforcer: If a recipe works really well, it adds a little "cheat sheet" to it to make it even better for next time.
The Antipattern Recorder: If a recipe fails, it doesn't just delete it; it writes down why it failed so the system knows what not to do next time.

3. The "Safety Inspector"

This is the most critical part. Before the filing cabinet accepts any new advice or throws out an old one, a Safety Inspector checks the math.

If the AI thinks a strategy works, the Inspector uses a calculator (symbolic tools) to verify the answer is actually correct.
If the math is wrong, the Inspector stops the system from saving that "bad advice."

This prevents the system from learning the wrong things, which is a common problem when AI tries to learn on its own.

4. The Results: Small, Fast, and Smart

The researchers tested this system on very hard math competitions (like high school Olympiads). They compared ISM to other methods:

The "Vanilla" Tutor: No memory at all. (Got stuck often).
The "Passive" Filing Cabinet: Just stored everything without cleaning or checking. (Got messy and slow).
The "Retrieval" System: Just searched through a giant pile of past examples. (Very slow and bloated).

The Winner: ISM.

It solved more problems correctly than any other method.
It remembered past lessons better (less "forgetting").
The Best Part: It did all this while storing 64% to 86% fewer strategies than the next-best system. While other systems were hoarding thousands of examples, ISM kept a tiny, highly efficient library of about 13 to 17 perfect strategies.

The Big Picture

The paper claims that you don't need to retrain the AI's brain to make it smarter. Instead, you can give it a small, self-cleaning, verified memory bank that learns from both its successes and its failures.

It's like giving a student a notebook where they don't just write down every homework problem, but instead write down the rules that worked, cross out the ones that didn't, and constantly organize the notebook so they can find the right rule instantly. The result is a student who gets smarter over time, even though their brain (the AI model) never changes.

Technical Summary: ISM – Self-Improving Strategy Memory for Continual Mathematical Reasoning

1. Problem Statement

The paper addresses the challenge of continual learning for Large Language Models (LLMs) under hard episodic resets. In this setting, an LLM with frozen parameters ( $\theta$ ) must solve a sequential stream of mathematical problems spanning shifting domains (e.g., Algebra, Geometry, Number Theory). Crucially, the model receives no context from previous episodes other than what is explicitly stored in an external memory repository ( $M_t$ ).

The core difficulty lies in the fact that standard LLMs lose insights gained from previous tasks once the context window resets. Existing approaches to mitigate this suffer from specific limitations:

Reflection-based methods (e.g., Reflexion) generate unstructured verbal feedback that is not easily reusable.
Retrieval-augmented systems store raw examples that grow without bound, relying on surface-level similarity and accumulating noise.
Skill-library systems (e.g., Voyager) expand monotonically without mechanisms to verify new entries, consolidate duplicates, or retire obsolete strategies.

The paper posits that for a frozen LLM to self-evolve in mathematical reasoning, it requires an external memory that is bounded, verifiable, and actively maintained rather than passively stored.

2. Methodology: Intelligent Schema Memory (ISM)

ISM is a self-evolving memory framework designed to retrieve strategies and actively regulate its repository via an autonomous self-improvement loop, all while keeping the underlying LLM frozen.

2.1. Core Architecture

The system operates through five interacting components:

Feature Extractor: Converts a problem $x_t$ into a structured ProblemFeatures object containing operator type, structural pattern, heuristic signatures, and a semantic embedding. It uses a hybrid approach (rule-based + LLM-based) reconciled by an agreement score.
Schema Bank: Stores compact strategy schemas. Each schema is dual-represented:
- Content: The strategy description, solution template, and heuristics injected into the solver prompt.
- Feature Hook: Structural and embedding-based retrieval signals that adapt online based on usage. This decouples what a schema knows from how it is discovered.
Two-Stage Retrieval:
- Stage 1 (Operator Filter): Hard filters candidates by matching the problem's operator type (e.g., filtering out geometry schemas for number theory problems).
- Stage 2 (Soft Scoring): Ranks remaining candidates using a weighted sum of structural, heuristic, quantity, embedding, and prior success rate similarities.
Frozen LLM Solver: Receives the problem and the retrieved schema's content. It operates with a "hard reset" (no prior context) but can optionally enter an agentic mode to invoke verifiable tools (symbolic computation, formal answer verification, schema lookup, failure search) if the initial answer fails a sanity check.
Memory Controller: Executes a self-improvement loop after each episode to update the bank.

2.2. Self-Improvement Mechanisms

The Memory Controller runs seven distinct mechanisms on independent schedules to maintain the bank's quality and size:

Self-Audit: Scores schemas as strong, neutral, weak, or unused based on success rates and usage counts.
Self-Correct: Rewrites the content of "weak" schemas (with sufficient usage) using an LLM and resets their success rate.
Self-Merge: Combines near-duplicate schemas (high embedding cosine similarity) into a single entry.
Self-Promote/Demote: Adjusts retrieval scores based on schema success rates.
Self-Prune: Removes unused schemas or those with persistent failure.
Self-Reinforce: Distills new positive heuristics from successful episodes and appends them to strong schemas.
Self-Antipattern: Extracts "mistakes-to-avoid" from failed episodes and records them in the content of associated schemas.

2.3. Conditional Schema Evolution & Verification

Evolution Gate: If a specific pattern of failures occurs (e.g., 3+ same-operator failures in the last 20 episodes), a synthesizer LLM creates a new general-family schema with a warm-started feature hook.
Verification Gate: Crucially, every memory update (merging, rewriting, reinforcing, synthesizing) is gated by symbolic verification. Updates only proceed if the underlying solution or shared use cases are verified by symbolic computation tools. This prevents the bank from reinforcing faulty generalizations.

3. Key Contributions

The authors identify three primary contributions:

ISM Architecture: A self-evolving memory system with an autonomous loop that retrieves strategies, learns symmetrically from both successes and failures, and gates all updates through symbolic verification.
Dual-Representation Design: A schema design that decouples content (strategy) from retrieval features (discovery), allowing retrieval accuracy to sharpen with experience without altering the underlying strategies.
Empirical Validation: Evaluation under a continual learning protocol with hard episodic resets on competition-level benchmarks, demonstrating superior performance with significantly reduced storage requirements.

4. Experimental Results

The system was evaluated on a 300-episode continual learning stream across two benchmarks: MATH-Hard (level-5 problems) and OlympiadBench (olympiad-level problems). It was compared against five baselines: Vanilla LLM, Static Schema, RAG-over-Examples, Reflexion, and Passive Schema Memory (ISM without self-improvement mechanisms).

Performance Metrics

Accuracy: ISM achieved the highest accuracy on both benchmarks (80.67% on MATH-Hard and 61.67% on OlympiadBench), outperforming the strongest baseline (Passive Schema Memory) by 2.00 percentage points on each.
Stability & Forgetting: ISM showed improved stability (retaining knowledge of past domains) and reduced forgetting compared to baselines. On OlympiadBench, ISM demonstrated positive Backward Transfer (BWT) of +0.03, indicating that learning new domains improved performance on old ones.
Memory Efficiency: ISM maintained a drastically smaller memory footprint:
- MATH-Hard: 17 schemas vs. 47 for Passive (64% fewer) and 300 for RAG/Reflexion.
- OlympiadBench: 13 schemas vs. 91 for Passive (86% fewer) and 300 for RAG/Reflexion.
- Overall, ISM used up to 23× fewer entries than retrieval-based systems.

Qualitative Findings

Case studies revealed that ISM's ability to synthesize new schemas after detecting repeated failure patterns allowed it to retrieve the correct strategy in subsequent episodes where baselines failed. The active maintenance (pruning/merging) prevented the "bloat" seen in passive systems.

5. Significance and Limitations

Significance

The paper argues that small, actively maintained, and verified strategy memories can support reliable continual mathematical reasoning in frozen LLMs. It demonstrates that external memory need not be a passive storage unit but can function as a self-correcting asset that improves over time without retraining the model. This addresses a fundamental AI question: how a frozen system can self-evolve while keeping accumulated knowledge bounded and verifiable.

Limitations

The authors explicitly note several limitations:

Evaluation Scope: Experiments used a single random seed and stream ordering; results are preliminary evidence rather than definitive estimates.
Ablation: The study evaluates the aggregate value of active maintenance (via the Passive baseline) but does not isolate the contribution of individual mechanisms (e.g., Self-Reinforce vs. Self-Merge).
Baseline Comparison: The comparison focuses on memory-oriented approaches and does not include search-heavy reasoning systems (e.g., Tree-of-Thoughts) or fully agentic frameworks.
Model and Domain Generalization: Results are limited to a single frozen LLM backbone and two mathematical benchmarks. Generalization to other domains (e.g., formal proofs, code) or different model sizes remains unverified.
Verification Constraints: The current verification gate relies on symbolic tools suitable for numerical/symbolic answers but is less applicable to proof-oriented tasks requiring complex reasoning.