Technical Summary: Model-Aware Skill Alignment for LLM Agents (MASA)

1. Problem Statement

Large Language Model (LLM) agents increasingly rely on retrieving externally curated procedural instructions, termed "skills," to solve long-horizon interactive tasks. Current skill library systems typically treat these libraries as model-agnostic, reusing identical skill formulations across backbones with vastly different capacities (e.g., 4B to 32B parameters).

The authors challenge this "one-size-fits-all" assumption through empirical evidence showing that skill effectiveness is strongly model-dependent. A skill formulation that benefits a specific backbone can actively degrade the performance of another. For instance, in the ALFWorld environment, a "Moderate" granularity skill set improved the performance of a Qwen3-4B model but reduced the performance of a Qwen3-8B model, which performed best with no guidance. Furthermore, the relationship between skill granularity and performance is non-monotonic; simply increasing detail does not linearly improve performance across all model scales or task types. This misalignment suggests that retrieved skills can distract rather than assist when their expression does not match the target model's default problem-solving strategies and capacity.

2. Methodology: MASA Framework

To address this, the authors propose MASA (Model-Aware Skill Alignment), a framework that adapts skill formulations to specific target backbones without modifying the agent's weights. MASA operates in two distinct stages:

Stage 1: Hierarchical Model-Conditioned Skill Evolution

This stage performs a search-based optimization to discover optimal skill libraries for a specific target model $F$. It utilizes a stronger "Teacher" LLM ($T$) and a structured Model Card ($M_F$) containing metadata, training provenance, and a capability profile (strengths/weaknesses) of the target model. The evolution is split into two phases:

1. General Skill Evolution (Iterative Hill Climbing):

   • Targets cross-task strategy principles (e.g., "verify action parsing").
   • The target model $F$ is rolled out across the full task suite.
   • The Teacher analyzes failure trajectories to produce structured failure attributions.
   • Guided by the Model Card and previous high-reward iterations, the Teacher rewrites the general skills.
   • Candidates are accepted only if they yield a higher adjusted reward (balancing success rate against "nothing-happens" stalling).

2. Task-Specific Skill Evolution (UCB-Driven Tree Search):

   • Targets domain-specific procedures (e.g., "check fridge before counter").
   • An independent tree search is run for each task type.
   • Nodes represent candidate skill sets; edges represent Teacher rewrites.
   • Selection: Uses UCB1 (Upper Confidence Bound) to balance exploration of new strategies and exploitation of high-reward nodes.
   • Expansion: The target model rolls out on specific task types; the Teacher rewrites skills based on failures.
   • Backpropagation: Rewards are propagated to update node values.

Stage 2: Model-Conditioned Skill Rewriter

To avoid the high computational cost of the search pipeline during deployment, MASA trains a lightweight Skill Rewriter ($f_\theta$).

• Training: The rewriter (instantiated as Qwen3-4B) is trained via Supervised Fine-Tuning (SFT) on the trajectories generated by the Stage 1 evolution pipeline. The input consists of the Model Card ($M_F$), an input skill set ($S_{in}$), and a task description ($d$). The target output is the evolved optimal skill set ($S^*_F$).
• Inference: At deployment, the rewriter adapts skills in a single forward pass, requiring no environment interaction or iterative search. It learns to transform unaligned skills into model-appropriate ones based on the target backbone's profile.

3. Key Contributions

1. Empirical Demonstration of Model-Dependency: The paper provides controlled experiments showing that the same skill library can benefit one backbone while degrading another. This finding invalidates the assumption that a single skill formulation serves all model scales equally.
2. MASA Framework: The authors introduce a two-stage framework combining iterative, model-conditioned search with a lightweight rewriter. This approach aligns skill expression with the specific capacity and behavioral patterns of the target model.
3. Comprehensive Evaluation: MASA is evaluated across three diverse environments (ALFWorld, WebShop, Search-augmented QA) and four Qwen3 backbones (4B, 8B, 14B, 32B).

4. Experimental Results

• Performance Gains: MASA consistently achieved the highest success rates across all backbones and environments.
  • In ALFWorld, MASA improved success rates by up to +25.8 points over the strongest baseline (Base Skill) for the Qwen3-8B model.
  • In WebShop, MASA significantly outperformed baselines, particularly for larger models (8B–32B) that previously suffered from excessive chain-of-thought generation and step-budget exhaustion.
• Efficiency: MASA reduced the average number of interaction steps required to complete tasks (e.g., reducing steps from 39.1 to 29.2 for Qwen3-8B in ALFWorld), indicating more precise action execution.
• Generalization (OOD): The trained Skill Rewriter demonstrated strong generalization to unseen task types and environments without additional search.
  • When trained on Search and WebShop data, the rewriter outperformed a one-shot teacher adaptation (DS-Adapter) on held-out ALFWorld tasks.
  • The rewriter (4B parameters) consistently surpassed the performance of a much larger Teacher LLM (DeepSeek-V4) used for one-shot adaptation, achieving this at a fraction of the inference cost.

5. Significance and Claims

The paper argues that skill libraries should be treated as model-aware artifacts rather than universal resources. The authors claim that:

• Alignment is Critical: Proper alignment of skill expression with model capacity unlocks capabilities that generic skills cannot access.
• Cost-Effective Adaptation: The MASA-Rewriter allows for the amortization of expensive search-time optimization into a single forward pass, enabling rapid adaptation to new backbones without environment rollouts or manual prompt engineering.
• Infrastructure Shift: The work positions skill alignment as a necessary infrastructure layer for deploying heterogeneous LLM agents, suggesting that even compact models can exhibit behaviors traditionally associated with frontier-scale systems when their skills are correctly aligned.

The authors maintain modesty regarding limitations, noting that their empirical evidence is currently restricted to the Qwen3 family and that the evolution pipeline relies on environments with automatic reward signals. They suggest future work could extend the framework to more model families and domains lacking built-in success/failure signals.