Technical Summary: CURATEEVO

Problem Statement

Large Language Model (LLM) agents require post-training methods capable of improving long-horizon decision-making using environment feedback. While data quality is a known determinant of performance in agentic post-training, existing data curation pipelines suffer from two primary limitations:

1. Curation Neglect: Most methods focus primarily on data augmentation (collecting more failed trajectories) while neglecting the critical operations of filtering and refining data. This often results in the inclusion of low-quality or redundant data, increasing training costs without proportional performance gains.
2. Adaptation Rigidity: Existing pipelines rely on fixed data processing rules. These static strategies struggle to adapt to out-of-distribution downstream applications or evolving failure modes, as they cannot dynamically adjust to the specific weaknesses of the current agent policy.

Methodology: CURATEEVO

The authors propose CURATEEVO, a failure-driven dynamic evolution framework for agentic post-training data curation. Unlike traditional approaches that treat curation as a static preprocessing step, CURATEEVO treats the curation strategy as executable code that is iteratively rewritten based on environment feedback.

Core Framework

The framework operates in multi-epoch evolution cycles:

1. Representation: The curation strategy $\rho$ is represented as executable code that transforms a fixed raw corpus ($D_{raw}$) into three resources:
   • Supervised Fine-Tuning data ($D_{SFT}$).
   • Reinforcement Learning data ($D_{RL}$).
   • An inference-time memory bank ($M_{mem}$).
2. Training & Evaluation: An agent policy is trained from a fixed base model using a fixed recipe (SFT + GRPO) on the curated data. The policy is then evaluated on a held-out development set ($Q_{dev}$).
3. Feedback Collection: Failed trajectories from the development set are collected, along with statistics on the current curated data (scale, turn distribution, redundancy).
4. Code Evolution: An LLM-based code-evolution agent analyzes the failed trajectories to diagnose recurring failure modes (e.g., incorrect tool selection, weak planning, memory mismatches). It then rewrites the curation code ($\rho$) to address these issues.

Optimization Objective

The evolution process optimizes a cost-aware objective function $J(\rho) = P(\rho) - \lambda C(\rho)$, where:

• Effectiveness ($P(\rho)$): Measured by the downstream performance of the agent trained with the curated resources. The evolution agent first targets this by augmenting data to cover missing capabilities, filtering misleading examples, and refining weak trajectories.
• Efficiency ($C(\rho)$): Measured by the normalized training-turn cost. After addressing effectiveness, the agent prunes redundant turns, removes low-utility samples, and truncates overly long trajectories to reduce the training scale while preserving critical supervision signals.

If an evolved strategy fails to improve the objective, the process rolls back to prevent error cascading.

Key Contributions

1. Dynamic Evolution Framework: The proposal of CURATEEVO, which shifts data curation from a static pipeline to a dynamic, code-based evolution process driven by agent failures.
2. Joint Optimization: The framework jointly considers data augmentation, filtering, and refinement, allowing for performance improvements while simultaneously reducing training costs.
3. Adaptability: By dynamically revising curation code based on evaluation feedback, the method adapts to out-of-distribution applications and specific failure modes, unlike fixed pipelines.
4. Resource Construction: The method explicitly constructs not just training data (SFT/RL) but also an inference-time memory bank, creating a holistic resource set for agent training.

Experimental Results

The authors evaluated CURATEEVO on ACEBench-Agent, BFCL-V4, and $\tau^2$-Bench under two settings: Labeled Data (human-annotated trajectories) and Wild Data (real user interaction logs).

• Performance Gains:
  • In the Labeled Data setting, CURATEEVO improved average scores by 3.2 points over the strongest prior baselines.
  • In the Wild Data setting, it improved average scores by 2.7 points.
  • Notably, CURATEEVO using a 4B parameter model (QWEN3-4B) outperformed several prior methods using 8B models, suggesting that adaptive curation can be as impactful as increasing model size.
• Efficiency: CURATEEVO reduced token overhead by approximately 48% and wall-clock overhead by 50% per 1,000 retained training turns compared to prior baselines. This efficiency stems from evolving code over a fixed corpus rather than relying on expensive environment interactions or simulated users.
• Compatibility: The curated resources produced by CURATEEVO are compatible with different post-training recipes (e.g., AgentGym-RL, ProRL-Agent), yielding an additional average improvement of 21.3 points when combined with these methods.
• Ablation Studies:
  • Removing the effectiveness-oriented revision caused the largest performance drop (~7 points), confirming that diagnosing failure modes is the primary driver of success.
  • Removing the efficiency-oriented revision also caused a consistent drop, indicating that pruning redundancy improves data quality, not just reduces scale.
  • All three resources (SFT, RL, Memory) were found to be complementary.

Significance and Claims

The paper claims that adaptive, feedback-driven data curation is a practical and effective direction for improving LLM agents. The significance of CURATEEVO lies in its ability to:

• Transform the curation process from a "fixed pre-processing step" into an iterative, failure-aware optimization loop.
• Demonstrate that filtering and refining are as critical as augmentation for agentic post-training.
• Show that cost-aware curation can reduce training overhead while simultaneously improving downstream performance, making it suitable for real-world agent applications where data may be noisy or imperfect.

The authors position CURATEEVO not as a replacement for specific policy optimization algorithms, but as a modular data-side enhancement that improves the quality and efficiency of the training resources ($D_{SFT}$, $D_{RL}$, $M_{mem}$) fed into any agentic post-training pipeline.