🤖 machine learning

SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolution

SkillRise is a unified reinforcement learning framework that enables large language model agents to evolve and reuse transferable skills across distinct tasks by organizing them into progressive sequences and employing a single policy to simultaneously solve tasks and curate an evolving skill document, achieving superior performance and efficiency compared to existing multi-stage or independent-task approaches.

Original authors: Zhiyuan Yao, Yuxin Chen, Zhengxi Lu, Zishan Xu, Yueqing Sun, Yifu Guo, Yuquan Lu, Zhengzhou Cai, Kangning Zhang, Zhuowen Han, Zi-Han Wang, Ziang Ye, Qi Gu, Xunliang Cai, Weiwen Liu, Yongliang Shen

Published 2026-07-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhiyuan Yao, Yuxin Chen, Zhengxi Lu, Zishan Xu, Yueqing Sun, Yifu Guo, Yuquan Lu, Zhengzhou Cai, Kangning Zhang, Zhuowen Han, Zi-Han Wang, Ziang Ye, Qi Gu, Xunliang Cai, Weiwen Liu, Yongliang Shen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to play a video game. In the old days, if the robot failed level one, it would forget everything, restart level two from scratch, and hope for the best. It was like trying to learn to ride a bike by falling off, getting up, and immediately trying to ride a different bike without remembering how you fell. This is how many computer programs called "AI agents" used to work: they treated every new challenge as a brand-new, isolated event, wasting all the lessons they learned on the previous ones.

But what if the robot could keep a "cheat sheet" or a "diary" that got smarter every time it played? This is the heart of a field called Reinforcement Learning, where AI learns by trying things, getting rewards for success, and learning from mistakes. Recently, scientists have been trying to teach these agents to not just solve a single puzzle, but to learn general "skills" that help them solve many different puzzles. The big question is: How do you teach an AI to write down its own lessons in a way that actually helps it later, without getting confused by the specific details of the first puzzle?

Enter SkillRise, a new method that acts like a super-smart tutor for AI agents. Instead of making the agent forget its past, SkillRise organizes tasks into a "level-up" sequence, moving from easy to hard. As the agent plays, it does two things at once: it tries to solve the current level, and it acts as a "curator" to update its own skill diary. Think of it like a video game character who, after beating a boss, doesn't just move to the next level but immediately writes a new entry in their journal: "Hey, remember how that boss only attacked when you were on the left? Next time, stay on the right."

The magic of SkillRise is how it rewards the AI. It gives points for solving the current task, but it also gives points for how well the "diary entry" helps the agent solve future tasks. This encourages the AI to write down general rules (like "stay on the right") rather than specific details (like "the boss was named Bob"). The researchers tested this on three different "worlds": a house where the agent has to clean and organize (ALFWorld), an online store where it has to buy specific items (WebShop), and a science lab where it runs experiments (ScienceWorld).

The results were impressive. SkillRise didn't just get better at the specific tasks it was trained on; it learned a way of learning. When the researchers gave it longer sequences of related tasks to practice on, the agent got even better, suggesting it was truly reusing its skills rather than just memorizing answers. In fact, it outperformed other top methods by a significant margin—improving success rates by up to 8.5 percentage points. Perhaps most surprisingly, even though it was trained on different tasks, it was still better at retrying the same task over and over than methods designed specifically for that. It also did all this much faster and with less computing power than older methods that required complex, multi-step pipelines to manage memory.

In short, SkillRise suggests that if you want an AI to get smarter, don't just let it practice; teach it to keep a running, evolving notebook of its own wisdom. By separating the act of "doing" from the act of "writing down what we learned," and by rewarding the agent for how useful that writing is for future challenges, we can build agents that don't just solve problems, but actually evolve their own strategies to become better problem-solvers over time.

Technical Summary: SkillRise

Problem Statement

Large Language Model (LLM) agents frequently encounter streams of related yet distinct tasks that share underlying regularities and reusable solution patterns. However, standard agentic Reinforcement Learning (RL) typically treats each task as an independent episode, discarding acquired experience and forcing the agent to re-explore from scratch. Existing approaches to skill learning suffer from specific limitations:

Instance-Specificity: Methods that repeatedly attempt the same task often produce knowledge that is specific to that instance, failing to generalize across different task instances.
Pipeline Entanglement: Approaches using external skill banks or multi-stage pipelines (involving separate extraction, retrieval, and execution modules) struggle with credit assignment. Because downstream performance depends on the interplay of all these stages, it is difficult to attribute success or failure specifically to the quality of the extracted skills.
Lack of End-to-End Optimization: There is a need for a unified framework that can jointly learn to solve tasks and curate transferable skills without complex external management pipelines.

Methodology: SkillRise

The authors introduce SkillRise, a unified, end-to-end reinforcement learning framework designed for cross-task skill evolution. The framework operates on a sequence of related but distinct task instances, organized from simple to challenging.

1. Cross-Task Sequence Construction

Tasks are grouped by environment-provided family metadata (e.g., product categories in WebShop, object types in ALFWorld) and ordered by difficulty. This creates a sequence $x = (x_1, \dots, x_K)$ where experience from earlier tasks is intended to support later ones.

2. Single-Policy Rollout with Skill Evolution

A single policy $\pi_\theta$ alternates between two roles within the sequence:

Task Solving: The agent solves the current task $x_i$ using an evolving textual skill document $S_{i-1}$ (which summarizes skills from previous tasks) as context.
Skill Curation: Upon task completion, the same policy rewrites the skill document $S_{i-1}$ $S_{i - 1}$ into a new version $S_i$ $S_{i}$ . This process involves:
- Preserving useful skills from the previous document.
- Consolidating successful procedures or failure modes revealed in the current trajectory.
- Removing instance-specific details (abstraction) to ensure reusability.
- The revised document $S_i$ becomes the sole information channel for the next task $x_{i+1}$ .

3. Decoupled Credit Assignment

To optimize both roles effectively, SkillRise employs a temporally aligned, decoupled credit assignment mechanism:

Task Solving: Supervised solely by the immediate reward $r_i$ of the current task.
Skill Curation: Supervised by the discounted sum of rewards from subsequent tasks ( $\sum_{j=i+1}^K \gamma^{j-i} r_j$ ). This ensures the curation policy is optimized for its ability to transfer value to future tasks, not just the current one.
Optimization: The framework uses a group-relative advantage approach, comparing returns across independent trials sharing the same task position and phase (solve vs. curate) to compute advantages, which are then used in a clipped objective function (similar to PPO) to update the policy.

Key Contributions

Formulation: The paper formulates cross-task skill learning over ordered sequences of distinct task instances, where later-task outcomes directly evaluate the transferability of skills curated from earlier interactions.
Framework: Development of SkillRise, an end-to-end RL framework that uses a single policy for both solving and curation, coupled with decoupled cross-task credit assignment and role-aware group-relative optimization.
Efficiency: The design eliminates the need for external skill-management pipelines (e.g., separate retrieval modules or teacher models), offering a compact training paradigm.

Experimental Results

The authors evaluated SkillRise on three interactive text-based benchmarks: ALFWorld, WebShop, and ScienceWorld, using Qwen3-1.7B and Qwen3-4B models.

Overall Performance: SkillRise achieved the strongest Pass@1 performance across all benchmarks, outperforming the strongest baseline (GiGPO) by 2.3 to 8.5 percentage points.
- ALFWorld: 85.9% (vs. 83.6% for GiGPO).
- WebShop: 84.4% (vs. 77.3% for GiGPO).
- ScienceWorld: 54.6% (vs. 46.1% for GiGPO).
Within-Task Generalization: Despite being trained on sequences of distinct tasks, SkillRise outperformed methods trained explicitly on repeated attempts of the same task (e.g., LaMer) when evaluated on Pass@2 and Pass@3 for held-out tasks. This suggests the learned curation policy captures a general strategy for converting interaction outcomes into actionable skills.
Cross-Task Test-Time Scaling: Performance improved monotonically as the length of the related task sequence ( $K$ ) increased at test time (e.g., from 83.6% at $K=2$ to 87.5% at $K=6$ on ALFWorld). Competing methods showed no such growth, indicating SkillRise effectively reuses and refines transferable skills across tasks.
Efficiency: Compared to multi-stage pipelines like RetroAgent and SkillRL, SkillRise achieved comparable or superior success rates while requiring only 1/6th of the running time, as it removes the overhead of external memory maintenance and retrieval.

Significance and Claims

The paper claims that SkillRise provides a simple and efficient training paradigm for LLM agents to extract, refine, and reuse transferable skills across tasks. Its primary significance lies in demonstrating that:

Cross-task skill learning can be achieved through a unified, end-to-end policy without complex external pipelines.
Decoupling the supervision signals for solving (current outcome) and curation (downstream outcomes) is critical for learning transferable regularities.
Agents can exhibit "test-time scaling" where performance improves with longer sequences of related tasks, even when each task is attempted only once, suggesting a robust mechanism for self-improvement through experience accumulation.

The authors note limitations, including the current reliance on task-family metadata for sequence construction and the restriction of experiments to models up to 4B parameters and text-based benchmarks.