🤖 AI

Coalition-Aware Skill Reliability for Self-Evolving Agents

This paper addresses the unresolved reliability of accumulated skills in self-evolving LLM agents by identifying coalition pollution and cross-domain utility reversal as key failure modes, and proposes Coalition-Aware Skill Selection (CASS) and Unsupervised Skill-Masked Coalition Optimizer (u-SMCO) to enhance task performance and generalization through reliability-aware skill management.

Original authors: Qiyan Zhao, Xiaofeng Zhang, Bo Liu, Minda Chen, Wei Xiong, Jingyang Chen, Guanting Ye, Wenhao Yu, Xiaosong Yuan, Shijie Han, Da-Han Wang, Jianmin Ji, Fei Huang, Xu-Yao Zhang

Published 2026-08-25

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Qiyan Zhao, Xiaofeng Zhang, Bo Liu, Minda Chen, Wei Xiong, Jingyang Chen, Guanting Ye, Wenhao Yu, Xiaosong Yuan, Shijie Han, Da-Han Wang, Jianmin Ji, Fei Huang, Xu-Yao Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a digital assistant that learns from its own mistakes and successes, building a personal library of "skills" to solve new problems. These skills are not just raw memories of what happened, but distilled, reusable instructions—like a chef learning a specific technique for chopping onions that can be applied to any recipe. In the world of artificial intelligence, researchers have been teaching large language models to do exactly this: gather experiences, turn them into structured tools, and store them in a "skill bank" to be called upon later. The hope is that as the agent accumulates more skills, it becomes smarter and more capable, much like a human expert who gets better with years of practice.

However, a fundamental question has remained unanswered: does having a larger library of skills actually make the agent better? Or could the agent be hoarding tools that are useless, or even harmful, in certain situations? A new study by researchers from several institutions, including the Chinese Academy of Sciences and the University of Science and Technology of China, investigates this precise issue. They discovered that simply adding more skills to an agent's memory does not guarantee improvement. In fact, the way skills work together is far more complex than previously thought, and the current methods for deciding which skills to keep are often blind to hidden problems.

The researchers began by examining how these self-evolving agents decide whether to accept a new skill into their bank. Typically, an agent tries a new skill, sees if the overall performance improves, and if the score goes up even slightly, the skill is kept. The team realized this approach is flawed because it looks only at the final score, ignoring how the new skill interacts with the existing ones. They found that a new skill might be accepted because the overall score improved, but that improvement might actually be caused by an older, reliable skill that happens to be in the bank at the same time. The new skill itself might contribute nothing, or even drag the performance down, but its negative effect is hidden by the success of its partners. The researchers call this "coalition pollution." It is like a team where a new member is hired because the team's total output went up, not realizing that the new member did nothing, and the boost came entirely from an existing colleague who was already working hard. The new hire is kept, polluting the team with a member who adds no value and might even cause friction later.

The study also uncovered a second problem that occurs when an agent moves from one type of task to another. An agent might learn a skill that works perfectly in a source environment, such as answering questions about long stories, and then carry that same skill into a new environment, like solving logic puzzles. The researchers found that skills which are helpful in the first setting can become harmful in the second. They call this "cross-domain utility reversal." A tool that helps a chef chop vegetables efficiently might be a disaster if the chef tries to use that same chopping motion to assemble a delicate watch. The agent, however, does not know this; it assumes that because the skill worked before, it will work again. The study showed that without intervention, agents often carry these harmful skills into new domains, where they actively degrade performance.

To fix these issues, the team developed two new methods. The first, designed to stop coalition pollution, changes how the agent decides to keep a new skill. Instead of just looking at the final score, the new method tests the new skill against many different combinations of existing skills. It asks: "Does this skill help when it is paired with Skill A? Does it help when paired with Skill B?" By averaging these results, the system can tell if the skill is genuinely useful or if it is just riding on the coattails of others. This ensures that only skills that truly contribute to the team's success are added to the bank.

The second method addresses the problem of skills becoming harmful in new environments. Since agents often do not have labeled data to tell them if a skill is working in a new domain, the researchers created a way to test skills without needing answers. They use the agent's own ability to find the right information as a test. If removing a specific skill makes the agent better at finding the right information in the new environment, that skill is likely harmful and is masked, or turned off. This allows the agent to clean up its library of skills automatically, even when it is exploring a completely new territory without a teacher to guide it.

The researchers tested these methods on four different benchmarks, ranging from complex reading comprehension tasks to simulated robot navigation in virtual worlds. In every case, the agents using the new methods performed better than those using standard techniques. They solved more tasks correctly and generalized their skills to new situations more effectively. The study also showed that by focusing on how skills work together, the agents became less sensitive to random fluctuations in their training scores, leading to more stable learning.

The core finding of this work is that the reliability of a skill is not an inherent property of the skill itself. A skill is not simply "good" or "bad." Its value depends entirely on the company it keeps in the skill bank and the specific environment where it is being used. A skill that is a hero in one context can be a villain in another, and a skill that seems useless on its own might be essential when paired with the right partners. By recognizing that skills function as a team rather than as isolated tools, the researchers have provided a way to build agents that are not just smarter, but more trustworthy and adaptable. This shift in perspective moves the field away from simply accumulating more data and toward understanding the complex relationships that make learning truly effective.

Technical Summary: Coalition-Aware Skill Reliability for Self-Evolving Agents

Problem Statement

Self-evolving agents based on Large Language Models (LLMs) increasingly rely on "skill banks"—collections of structured artifacts distilled from interaction trajectories—to learn from past experiences. While existing research has focused on the operational lifecycle of these skills (acquisition, evolution, retrieval, and composition), a fundamental question regarding mechanistic reliability remains unresolved: Do the skills accumulated in an agent's bank actually make positive contributions when invoked?

The authors identify that current skill evolution mechanisms, which typically rely on "outcome-only gates" (accepting a new skill if the aggregate task reward increases), fail to account for the context-dependent nature of skill utility. Through systematic audits, the paper reveals two critical reliability failure modes:

Coalition Pollution: During skill evolution, an agent may accept a new skill because the total bank performance improves, yet the new skill itself contributes negligibly or negatively. The observed gain is actually driven by incumbent skills or specific synergistic interactions, masking the unreliability of the newly admitted skill.
Cross-Domain Utility Reversal: Skills beneficial in a source domain may become detrimental when transferred to a target domain. Existing methods assume source-domain utility is invariant, but the audit shows that skills can reverse their effects (from helpful to harmful) upon domain transfer, degrading performance.

Methodology

To address these failures, the authors propose a Skill Mechanistic Reliability Audit (SMRA) and two corresponding interventions:

1. Skill Mechanistic Reliability Audit (SMRA)
SMRA evaluates skill reliability not as an intrinsic property but as a joint function of the skill, the surrounding skill bank (coalition), and the deployment domain. It utilizes Shapley values to measure marginal contributions:

Full-bank marginal contribution ( $\Delta$ ): Measures the performance drop when a specific skill is removed from the full bank.
Shapley value ( $\phi$ ): Averages marginal contributions across all possible subsets of the remaining skills to estimate a skill's expected contribution under varying coalitions.

2. Coalition-Aware Skill Selection (CASS)

Goal: Prevent coalition pollution during the skill evolution phase.
Mechanism: Instead of relying solely on aggregate reward ( $r_{out}$ ), CASS augments the acceptance criterion with a coalition-aware signal.
Process: When evaluating a candidate skill, CASS uses Monte Carlo sampling to construct diverse skill coalitions from the resulting bank. It calculates the knockout margin for each sampled coalition $S$ by dividing the joint performance loss of removing the entire subset $S$ by the subset size, defined as $\Delta_S(B) = \frac{V(B) - V(B \setminus S)}{|S|}$ . This approach charges the joint loss of removing $S$ equally to its members.
Decision Rule: The final score ( $\sigma$ ) combines the average of these sampled coalition margins ( $\bar{\Delta}_N$ ) with the online outcome reward ( $r_{out}$ ):
$\sigma(B) = \bar{\Delta}_N(B) + \lambda r_{out}(B)$
A candidate bank is retained only if this score exceeds the best score seen so far. This ensures that skills are selected based on their reliability within a coalition, not just their ability to boost a noisy aggregate metric.

3. Unsupervised Skill-Masked Coalition Optimizer (u-SMCO)

Goal: Mitigate cross-domain utility reversal during deployment.
Mechanism: A label-free method that masks skills whose utility reverses in the target domain.
Process: Using unlabeled queries from the target domain, u-SMCO scores each skill based on its contribution to retrieval quality. It greedily masks (removes) skills whose exclusion improves retrieval performance on the target data.
Significance: This approach requires no target-domain task labels, making it applicable at deployment time without supervision.

Key Contributions

Identification of Failure Modes: The paper formally defines and demonstrates "coalition pollution" and "cross-domain utility reversal" as distinct mechanistic failures in self-evolving agents.
Theoretical Insight: The authors prove that outcome-only gates are structurally blind to coalition-level interactions. A positive aggregate gain does not guarantee that every component skill is contributing positively (Theorem 1: Outcome-only non-identifiability).
Methodological Interventions:
- CASS: Introduces a sampling-based approach to estimate coalition-level reliability by averaging the knockout margins of sampled skill subsets (normalized by subset size), rather than computing per-skill Shapley values.
- u-SMCO: Proposes a label-free, retrieval-quality-based masking strategy to adapt skill banks to new domains.
Unified Reliability View: The work establishes that skill reliability is a contextual property ( $R(s; B, D)$ ) dependent on the skill, the bank, and the domain, rather than an intrinsic attribute of the skill itself.

Experimental Results

The proposed methods were evaluated on four benchmarks: LoCoMo, LongMemEval, HotpotQA, and ALFWorld.

Performance: CASS and u-SMCO consistently outperformed strong self-evolving baselines (including CoN, ReadAgent, MemoryBank, A-MEM, Mem0, MemoryOS, and MemSkill) in both task performance (F1, LLM-as-Judge scores) and cross-domain generalization.
Specific Gains:
- On LoCoMo, the proposed method achieved an F1 of 41.48 and an L-J score of 56.58, surpassing the previous best (MemSkill) which scored 39.45 F1 and 54.78 L-J.
- On ALFWorld (Unseen), the method achieved a Success Rate (SR) of 83.22 with 19.93 steps, outperforming all baselines.
Robustness: The coalition-conditioned modeling reduced sensitivity to noisy outcome-reward fluctuations during reinforcement learning, providing more stable training signals.

Significance and Claims

The paper claims to establish skill reliability as a joint property of skills, banks, and deployment domains, shifting the paradigm from viewing skills as static, intrinsic assets to dynamic, context-dependent units.

Beyond Accuracy: The work argues that accuracy alone is insufficient; understanding the mechanistic contribution of skills within a coalition is necessary for true self-evolution.
Limitations of Isolation: The authors emphasize that isolation-based skill evaluation is fundamentally limited because it cannot detect coalition pollution or domain-specific reversals.
Practical Impact: By introducing audit toolkits and interventions that do not require target-domain labels (u-SMCO), the paper provides a pathway for deploying more reliable, self-evolving agents in open-ended, dynamic environments.

The authors conclude that future reliable skill evolution must treat the skill coalition as the central unit of analysis, rather than individual skills in isolation.