Technical Summary: Knowledge-Adaptive Context Engineering (KACE) for Mathematical Reasoning

Problem Statement

Context engineering has emerged as a viable strategy to enhance the performance of frozen Large Language Models (LLMs) without updating model weights. However, for complex mathematical reasoning, existing approaches face a critical bottleneck: monolithic context accumulation. When feedback and learned lessons are aggregated into a single, growing prompt, two issues arise:

1. Context Bloat: Irrelevant guidance eventually crowds out useful information, degrading performance.
2. Knowledge-Store Ceiling: The reusable knowledge base is strictly bounded by the size of the active prompt.

Current methods (e.g., ACE, GEPA) often conflate storage (what is learned across runs) with usage (what is included in the prompt for a specific problem). Consequently, every problem must read the entire accumulated knowledge store, regardless of whether the information is relevant to the specific difficulty or domain of the task. Additionally, while adaptive compute allocation (varying inference effort based on difficulty) is a known effective strategy, it is rarely combined with structured, difficulty-aware knowledge retrieval.

Methodology: KACE Framework

KACE introduces a framework that decouples the storage of learned knowledge from its usage by organizing knowledge into a structured Epistemic Tree and retrieving it conditionally based on an empirical difficulty signal.

1. The Epistemic Tree (Storage)

Instead of a single prompt, KACE constructs a persistent knowledge base $K$ consisting of typed cards (e.g., lemmas, invariants, decomposition patterns, verification rules). This tree is stratified along two axes:

• Difficulty Tiers ($T$): Easy (ES), Medium (MS), Hard (HS).
• Epistemic Domains ($D$): Mathematical domains (e.g., geometry, combinatorics).

The tree is a block-structured object $K = \{K_b\}_{b \in B}$, where each block $b = (t, d)$ contains cards visible only at that specific difficulty tier and domain. Universal cards are stored separately and visible across tiers.

2. Locked Tiered Self-Consistency (The Classifier & Solver)

KACE employs a locked tiered self-consistency pipeline that serves dual purposes: solving the problem and classifying its difficulty. The pipeline proceeds in escalating stages:

• Easy Solver (ES): 2 attempts. Exits on 2/2 unanimous agreement. Reads no learned cards.
• Medium Solver (MS): 3 attempts. Exits on 2/3 majority agreement. Reads cards from the $(MS, d)$ node.
• Hard Solver (HS): 5 attempts. Exits on 2/5 plurality agreement. Reads cards from the $(HS, d)$ node.
• Fallback: If no tier exits, a pool of 10 attempts is used.

The tier at which the system exits provides an empirical difficulty signal. This signal, combined with a coarse domain classifier, projects the problem to a specific block in the Epistemic Tree. Crucially, easy problems exit card-free, while escalated problems retrieve only the matching branch of the tree, preventing context bloat.

3. Offline Construction (Learning)

The Epistemic Tree is constructed offline from training traces using a two-phase process:

• Phase 1 (ADD): A block-local learning loop. For each block $(t, d)$, the system analyzes failed traces where the solver exited at that tier. A teacher model distills "residuals" (missing knowledge) into new cards, which are added to the corresponding block. This follows a block-coordinate minimization approach, optimizing local pieces of the tree without global re-optimization.
• Phase 2 (REFINE): A curator evaluates the provisional tree under the induced distribution of the updated policy. It edits, relocates, or deprecates cards to ensure they remain useful and non-redundant when the solver actually uses them. This addresses the distribution shift problem common in iterative learning.

Key Contributions

1. Difficulty- and Domain-Stratified Epistemic Tree: KACE stores reusable mathematical reasoning knowledge outside the active prompt as typed cards organized by difficulty and domain. This separates the durable knowledge body from the small context slice read by the solver.
2. Locked Tiered Self-Consistency as a Difficulty Classifier: The tiered pipeline acts as a compute-equivalent alternative to fixed Best-of-N self-consistency (matching or exceeding Best-of-5 on three benchmarks) while simultaneously providing a calibrated empirical-difficulty signal. This signal preserves the model's own solve-rate ordering on 78% of AIME 2025 pairs and 80% of MATH-HARD pairs, eliminating the need for a separate, trained difficulty classifier.
3. Empirical Evidence on Hard Reasoning: The paper demonstrates that separating storage from usage via conditional retrieval significantly outperforms monolithic context accumulation.

Experimental Results

Experiments were conducted on three challenging benchmarks: AIME 2025, MATH-HARD (Level 4–5), and OlymMATH-EN (verifiable subset).

• Performance Gains: On AIME 2025, KACE achieved 62.2% accuracy. This represents a 10.4-point absolute gain over fixed Best-of-5 self-consistency (51.8%) and a 5.6-point gain over the strongest learned-context baseline, Tiered + GEPA (56.6%).
• Compute Efficiency: KACE achieves these gains at a comparable solver-call budget to Best-of-5. The tiered approach naturally exits early on easier problems, keeping the average number of calls close to 5.
• Ablation Studies:
  • Removing the domain axis (flattening to difficulty only) dropped accuracy from 62.2% to 54.4%.
  • Removing the difficulty axis (monolithic context) dropped accuracy to 51.6%.
  • Replacing distilled cards with raw solver reflections reduced accuracy to 59.3%, highlighting the value of teacher-distilled, compact knowledge.
  • Replacing the fixed tree projection with generic retrieval (LLM router or embedding) yielded lower performance (~56%), suggesting that the structured difficulty-domain projection is superior to generic retrieval for this task.

Significance and Claims

The paper claims that the primary bottleneck in context engineering for hard mathematical reasoning is not the amount of knowledge stored, but the ability to expose the right part of that knowledge at the right time.

• Structural Separation: KACE demonstrates that treating learned context as a structured resource, conditionally read based on empirical difficulty and domain, is more effective than monolithic playbooks.
• Calibration without Training: The tiered self-consistency exit path serves as a robust, model-derived difficulty signal that requires no external training data or separate classifier, yet aligns closely with empirical solve rates.
• Auditability: By storing knowledge as natural-language cards organized by difficulty and domain, the system's "learned" behavior is inspectable and auditable, a property not present in weight-updated models.

The authors note limitations, including the reliance on repeatable error patterns for learning, the lack of automatic cross-benchmark transfer, and the observation that scaling the knowledge base beyond training-trace failures (e.g., via "imagination") leads to overfitting without a stronger verifier. The work positions KACE as a template for studying context as a structured, conditionally-read resource rather than a static accumulation of text.