Technical Summary: SkeMex – Experience Makes Skillful

1. Problem Statement

The paper identifies a critical gap in the deployment of medical Large Language Model (LLM) agents. While current medical LLMs excel at static, single-turn question answering (e.g., MedQA, MedMCQA), real-world clinical decision-making is dynamic, multi-step, and requires continuous interaction, evidence gathering, and hypothesis revision.

Existing memory-augmented agents attempt to address this by storing past interactions, but they suffer from three primary limitations:

1. Redundancy and Noise: They often retain raw historical traces (entire trajectories) which are noisy, instance-specific, and difficult to govern, leading to redundant information accumulation.
2. Lack of Utility Awareness: They rarely distinguish which memories are truly useful for future reasoning. Low-quality or harmful entries accumulate over time, degrading performance.
3. Costly Adaptation: Many state-of-the-art methods couple memory improvement with policy training or self-distillation, requiring parameter updates. This is computationally expensive and risks catastrophic forgetting or weak transfer across domains.

The core challenge is enabling medical agents to accumulate compact, reliable, and reusable experience without modifying the backbone model weights, thereby supporting long-horizon clinical reasoning in a post-deployment setting.

2. Methodology: SkeMex

The authors propose SkeMex, a post-deployment self-evolution framework that improves medical agents through a skill-based memory without updating model weights. SkeMex treats interaction trajectories as sources of reusable experience, distilling them into structured "skills" that encode actionable reasoning and decision-making procedures.

Core Components

The framework operates on a Memory-based Markov Decision Process (M-MDP) and follows a closed-loop "Read–Write–Assess–Govern" lifecycle:

• Multi-Branch Skill Repository:
  Instead of storing raw trajectories, SkeMex organizes distilled skills into three hierarchical branches to separate abstraction levels:

  • General Branch ($M^{gen}$): Stores transferable reasoning strategies and broad clinical principles.
  • Task-Level Branch ($M^{task}$): Captures patterns tied to specific task families (e.g., differential diagnosis, treatment planning).
  • Action-Level Branch ($M^{act}$): Records operational knowledge for tool use (e.g., parameter formatting, specific tool invocation).

• Read: Value-Aware Retrieval:
  At the onset of an episode, the agent retrieves skills based on both semantic relevance and context-dependent utility. The retrieval score combines:

  • Semantic similarity to the current query.
  • Historical utility estimated from environment feedback within the specific clinical category.
  • Memory strength (temporal decay based on the Ebbinghaus forgetting curve).
    This ensures that only high-value, context-relevant skills are injected into the agent's prompt.

• Write: Trajectory-to-Skill Distillation:
  Not all interactions are written to memory. A gated trajectory buffer filters out infrastructure errors, mechanical repetitions, and trivial successes. Only informative trajectories (e.g., those revealing causes of misdiagnosis or successful multi-step reasoning) are retained.

  • Two-Pass Distillation: An analysis pass extracts reusable patterns and determines intent (CREATE, PATCH, or NONE). A mutation pass then generates a new skill draft or updates an existing one.
  • Review Gates: Drafts undergo novelty and quality checks to prevent redundancy and ensure they contain concrete situational triggers and steps rather than vague principles.

• Assess: Utility-Driven Valuation:
  Skills are not static; their value is continuously evaluated. SkeMex uses window-level valuation to aggregate feedback over multiple trajectories.

  • Advantage Estimation: Utility is updated based on the relative advantage of a trajectory ($A_\tau = r_\tau - \bar{r}$) rather than absolute reward, ensuring fair credit assignment across tasks of varying difficulty.
  • Credit Assignment: Positive adoption increases utility; harmful adoption decreases it. A risk-sensitive regularizer discourages high-utility skills from accumulating unsafe behaviors.

• Govern: Closed-Loop Repository Maintenance:
  Periodically, a governance operator manages the repository to maintain compactness and reliability:

  • Merge: Combines highly similar skills.
  • Deprecate: Removes low-utility or harmful entries.
  • Promote: Elevates consistently effective skills to "mature" status.
  • Capacity Control: Enforces limits on branch sizes to prevent unbounded growth.

3. Key Contributions

1. SkeMex Framework: A novel post-deployment self-evolution framework that enables medical agents to improve via skill-based memory without updating model weights, decoupling memory evolution from policy training.
2. Non-Parametric Reinforcement: Formulates skill-memory evolution as a non-parametric process where clinical feedback provides reward signals to estimate context-dependent utility, guiding both retrieval and repository governance.
3. Read-Write-Assess-Govern Lifecycle: Introduces a closed-loop mechanism that converts trajectories into reusable skills and maintains a well-governed memory repository through iterative filtering, distillation, valuation, and pruning.
4. Plug-and-Play Generalization: Demonstrates that SkeMex is a modular framework that consistently improves performance across diverse clinical tasks, generalizes effectively across different model backbones, and supports transferable skill memory across heterogeneous task settings.

4. Experimental Results

The authors evaluated SkeMex on nine diverse medical benchmarks, including interactive clinical environments (AgentClinic, LiveClin), knowledge-intensive reasoning (MedXpertQA, MMMU), and rubric-based evaluation (HealthBench, LiveMedBench).

• Offline Performance: SkeMex consistently outperformed representative memory-based agents and memory-free baselines.

  • Using the DeepSeek-V3.2 backbone, SkeMex improved the average score from 48.20% (ReAct) to 56.08% (+7.88 points), outperforming the strongest non-SkeMex memory baseline by 3.84 points.
  • Using the Qwen3.6-Plus backbone, it improved from 48.63% to 59.22% (+10.59 points).
  • In out-of-domain (OOD) settings, SkeMex showed superior transferability, achieving a +13.78 point gain over ReAct, significantly outperforming other memory methods which often suffered from negative transfer.

• Online Performance: In streaming task settings where the repository updates during interaction, SkeMex showed stable, continual improvement from epoch 1 to epoch 3, whereas several baselines regressed after memory updates.

• Ablation Studies:

  • Buffer Management: Removing the gating mechanism (filtering out noisy trajectories) caused the largest performance drop, confirming the necessity of selective trajectory retention.
  • Multi-Branch Structure: The full three-branch design (General, Task, Action) significantly outperformed single-branch or two-branch variants, highlighting the need for separated abstraction levels.
  • Utility Valuation: Removing category-aware reward normalization or harm clamping led to significant performance degradation, validating the importance of context-dependent utility estimation.

• Cross-Backbone Transfer: Skills distilled using DeepSeek-V3.2 successfully improved performance when applied to other backbones (Qwen3.6-Max, Kimi-2.6, GLM-5.1), demonstrating that the learned skills capture transferable procedural knowledge rather than model-specific patterns.

5. Significance and Claims

The paper claims that SkeMex offers a scalable step toward medical agents that can mature through continued clinical experience. By transforming raw interaction histories into an evolving, governed memory of structured skills, SkeMex enables:

• Experience-Driven Learning: Agents can accumulate and reuse reliable experience without the computational cost and risks associated with fine-tuning model weights.
• Robust Generalization: The structured skill abstraction and utility-guided retrieval reduce negative transfer, allowing agents to adapt to unseen clinical scenarios and diverse model backbones.
• Reliability: The closed-loop governance ensures that the memory remains compact, selective, and clinically dependable, filtering out harmful or redundant entries over time.

The authors position SkeMex not as a replacement for professional medical judgment, but as a decision-support tool that enhances the consistency and reliability of medical reasoning systems by leveraging accumulated, distilled experience.