Technical Summary: EvoDS

Problem Statement

While Large Language Model (LLM) agents have advanced automated data science, existing approaches face two fundamental limitations:

1. Static Action Sets: Current agents rely on predefined workflows and fixed action spaces. They treat tools as static primitives rather than learnable capabilities, forcing agents to repeatedly rediscover solutions via trial-and-error rather than abstracting successful behaviors into reusable skills.
2. Context Management Deficiencies: Data science workflows inherently require long-horizon, iterative reasoning. Existing agents lack principled mechanisms to manage the rapidly growing context arising from intermediate artifacts, evolving skills, and execution logs. This leads to "lost-in-the-middle" phenomena, degraded reasoning performance, and out-of-token failures, preventing systematic experience accumulation.

The core research question addressed is whether autonomous data science agents can be designed to not only execute tasks but also support open-ended exploratory analysis and systematically accumulate experience across iterations and tasks.

Methodology

The authors propose EvoDS, a self-evolving autonomous data science agent that integrates skill acquisition and context regulation within a hierarchical multi-agent architecture. The framework is optimized via a two-stage training scheme involving Supervised Fine-Tuning (SFT) and Agentic Reinforcement Learning (RL).

1. Hierarchical Multi-Agent Architecture

EvoDS decomposes complex data science workflows into atomic subtasks managed by specialized agents:

• Manager Agent: Acts as the global controller, responsible for high-level reasoning, task decomposition, and coordination. It maintains global memory (task objectives) and interacts with the environment using a minimal set of tools.
• Specialized Sub-Agents: Include a Cleaner, Featurizer, Modeler, Visualizer, and Debugger. Each operates within a localized action space ($A_i$) specific to its domain, maintaining local memory for subtask-specific context.
• Interaction: The Manager delegates subtasks to sub-agents. Sub-agents execute operations and return results. This decomposition reduces reasoning complexity and localizes long execution contexts.

2. Autonomous Skill Acquisition (ASA)

To enable self-evolution, EvoDS introduces a mechanism for agents to synthesize, validate, and reuse executable skills ($a = \langle n, d, c \rangle$, where $n$ is name, $d$ is description, $c$ is code). The process involves four stages:

• Synthesis: When a subtask exceeds existing capabilities, the agent generates a new executable skill via prompting.
• Verification: The synthesized skill is executed to ensure it produces valid outputs.
• Caching: Validated skills are stored in a synthesized skill repository ($\Delta A_i$).
• Expansion: To prevent action space bloat, skills are permanently added to the agent's action space only if their generation count exceeds a threshold ($\tau=3$), ensuring only frequently used, high-quality skills are retained.

3. Adaptive Context Compression (ACC)

To manage context growth, EvoDS employs a two-level compression strategy:

• Sub-Agent Level: Sub-agents distill raw execution results into concise summaries conditioned on the global task objective, filtering unnecessary details before returning results to the Manager.
• Manager Level: Unlike passive truncation, the Manager Agent possesses a dedicated summarization tool. It adaptively determines when to compress context based on the current reasoning state and task objectives, preserving task-critical information while suppressing irrelevant noise.

4. Agentic Optimization

EvoDS is trained using a two-stage framework:

• SFT (Stable Initialization): A teacher model (DeepSeek-V3.1) generates trajectories for both the Manager and sub-agents. These trajectories (main and sub-trajectories) are used to initialize coordinated behavior.
• Joint Reinforcement Learning (RL): The system uses Group Relative Policy Optimization (GRPO) to jointly optimize task performance, skill acquisition, and context management.
  • Rewards: Sub-agents receive rule-based rewards for subtask completion. The Manager receives a hybrid reward ($R_{main}$) balancing solution quality ($R_{outcome}$), subtask completion efficiency, and penalties for excessive context length ($P_{context}$) and interaction turns ($P_{turn}$).
  • Objective: The training objective aligns with an information bottleneck principle, minimizing task-irrelevant information while preserving critical signals.

Key Contributions

1. EvoDS Framework: A self-evolving agent that unifies Autonomous Skill Acquisition and Adaptive Context Compression within a hierarchical multi-agent system, enabling progressive skill expansion and active long-term context control.
2. Training Scheme: A two-stage multi-agent training strategy (SFT + Joint RL) that optimizes task execution, skill learning, and context regulation simultaneously, allowing the agent to improve with experience while remaining robust under token constraints.
3. Theoretical Insights:
   • Proof that the hierarchical architecture reduces tool-selection error probability compared to flat frameworks by decomposing large decision spaces.
   • Demonstration that the Manager's optimization objective is equivalent to an Information Bottleneck problem, theoretically justifying the balance between efficiency and decision quality.
4. Empirical Performance: Extensive experiments showing significant improvements over state-of-the-art open-source agents.

Experimental Results

The authors evaluated EvoDS on four benchmarks: DABench, DA-Code, ScienceAgentBench (SAB), and MLE-Dojo.

• Performance: EvoDS outperformed the strongest open-source baseline (DataMind-14B) by an average of 28.9% (9.5% absolute improvement), despite using a smaller model backbone (Qwen3-8B). It achieved the second-best average performance overall, surpassed only by ReAct with the proprietary o4-mini backbone.
• Long-Horizon Robustness: EvoDS eliminated out-of-token failures across all benchmarks, whereas baselines without context compression frequently exceeded token limits, especially on complex tasks like MLE-Dojo.
• Skill Reuse: Experiments demonstrated that synthesized skills could be effectively reused across tasks. The "w/ reuse" variant outperformed "w/o reuse" by 2.9% on DA-Code and 9.3% on ScienceAgentBench, with a 69% cross-task reuse rate for synthesized skills.
• Ablation Studies: Removing any core component (RL, hierarchical structure, ASA, or ACC) resulted in performance degradation. Notably, removing ACC led to the most severe drop in performance and a high rate of token-limit failures.

Significance and Claims

The paper claims that EvoDS represents a significant step toward scalable, adaptive, and autonomous data science systems. By moving beyond static workflows, EvoDS demonstrates that agents can:

• Accumulate Experience: Systematically convert successful problem-solving procedures into persistent, reusable skills.
• Manage Complexity: Effectively regulate long-horizon contexts through adaptive compression, enabling reliable execution in multi-stage pipelines without token exhaustion.
• Bridge the Gap: Narrow the performance gap between open-source and proprietary foundation models through architectural design and effective training strategies rather than relying solely on model scale.

The authors acknowledge that while EvoDS excels in procedural automation and skill accumulation, its performance on tasks requiring deep domain knowledge (e.g., complex scientific discovery) remains constrained by the underlying foundation model's knowledge, suggesting future work should focus on integrating stronger external domain knowledge.