Technical Summary: Harness-Aware Training for Evolvable Digital Avatar Agents

1. Problem Definition

The paper addresses the operational challenges of deploying AI-powered digital avatar streamers in live e-commerce. These agents must simultaneously satisfy three conflicting requirements: ultra-low latency (to maintain viewer engagement), high adaptability (to accommodate weekly changes in marketing strategies, compliance rules, and reply styles), and factual accuracy (avoiding hallucinations in public broadcasts).

A critical tension exists between the need for rapid business iteration and model stability. Traditional approaches rely on Fixed-Harness Supervised Fine-Tuning (SFT), where a model is trained on a static configuration of skills, tools, and prompts. However, the authors identify that this leads to surface-form overfitting: the model memorizes specific skill names, tool schemas, and prompt templates rather than learning to interpret the functional instructions. Consequently, when the "Harness" (the execution environment comprising Skills, Hooks, Prompts, and Tools) evolves—a process the authors term Harness Evolution—the fixed model becomes stale and fails to adapt.

Conversely, large zero-shot models possess the generalization to handle new configurations but suffer from latencies (often >11 seconds) that exceed real-time interaction budgets. Smaller, compact models are fast enough but lack the generalization to handle evolving environments without retraining. The paper posits that existing solutions like test-time adaptation violate latency constraints, while continual learning incurs unsustainable retraining costs for frequent updates.

2. Methodology: Harness-Aware Training (HAT)

To resolve this tension, the authors propose Harness-Aware Training (HAT), a methodology that treats the Harness state as an explicit variable within the training distribution rather than a fixed background assumption. The core innovation is Harness-State Augmentation (HSA), which applies task-preserving transformations to the environment during training to force the model to learn the function of the instructions rather than their surface form.

2.1 Harness-State Augmentation (HSA)

HSA perturbs the Harness configuration across five dimensions to prevent memorization:

1. Skill Identifiers: Randomly renaming skills, masking subsets, and rewriting descriptions.
2. Skill Content: Paraphrasing rules, masking subsets, and reordering them.
3. Tool Definitions: Renaming tools and rewriting descriptions while preserving function.
4. System Prompt: Reordering instruction blocks and perturbing numeric constraints within valid ranges.
5. Hooks: Simulating variations in retry behavior and message structures.

2.2 The Three-Stage Training Pipeline

The HAT pipeline consists of three distinct stages designed to build a compact, robust agent:

• Stage 1: HSA-SFT (Supervised Fine-Tuning with Augmentation): The model is trained on trajectories generated under diverse, augmented Harness configurations. This exposes the model to multiple realizations of the same task semantics, encouraging it to rely on functional meaning rather than fixed identifiers.
• Stage 2: General Domain On-Policy Distillation (General OPD): To counteract the "catastrophic forgetting" of general instruction-following capabilities often seen in domain-specific SFT, the model undergoes on-policy distillation using a general instruction dataset (Tulu3). The student model minimizes the KL divergence relative to a pre-trained base model, recovering general capabilities while retaining domain knowledge.
• Stage 3: HSA-RL (Agentic Reinforcement Learning with Augmentation): The policy is further optimized in a production-informed live-streaming simulator using Group Relative Policy Optimization (GRPO). Crucially, HSA is applied during the RL rollout generation. The model learns to navigate dynamic interaction loops, handle tool errors, and recover from failures under shifting environmental conditions. The reward signal includes Accuracy, Effectiveness, Tool Rationality, and Skill Selection, with a penalty for excessive Chain-of-Thought (CoT) length.

3. Key Contributions

The paper outlines four primary contributions:

1. Evolvable Harness Architecture: A decoupled system where Skills, Prompts, Hooks, and Tools are updated independently of model weights. This enables business logic to evolve via a "diagnose–edit–evaluate" loop without retraining, but creates the challenge of a "moving target" for the policy model.
2. Harness-Aware Training (HAT): A novel training paradigm that integrates the Harness state into the training distribution via HSA. This allows a latency-feasible compact model to track Harness changes without memorizing a single configuration.
3. Comprehensive Evaluation Framework: The construction of four complementary evaluation sets (Live-Stream QA, Harness-Variant QA, Tool/Prompt Robustness, and IFEval) totaling over 4,500 cases. This includes a human-aligned "Agent-as-a-Judge" calibrated against human annotations.
4. Practical Training Insights: Empirical findings revealing that HSA has a larger impact during the SFT stage than the RL stage, and that General OPD is critical for recovering instruction-following abilities lost during domain adaptation.

4. Experimental Results

The authors evaluated a 35B parameter model (Qwen3.6-35B-A3B) trained with HAT against base models, standard SFT baselines, and strong zero-shot models.

• Live-Stream QA Performance: The HAT model achieved an average score of 94.8, surpassing the base model (80.3) and the strongest zero-shot general LLM (93.0).
• Robustness to Evolution: On the Harness-Variant QA set (testing generalization to new configurations), HAT scored 94.6, significantly outperforming the base model (75.4).
• Generalization Preservation: Crucially, HAT maintained an IFEval score of 83.5, avoiding the 7.7-point drop typically observed in models trained with Fixed-Harness SFT (which dropped to 73.8). This demonstrates that HAT preserves general instruction-following capabilities.
• Latency and Efficiency: Deployed on a single NVIDIA H20 GPU with Multi-Token Prediction (MTP) enabled, the agent achieved a P50 latency of 3.407s and a P95 latency of 8.114s, meeting the strict <15s requirement for live streaming. MTP increased decoding throughput by 1.69x at concurrency 1.
• Human Evaluation: In a blind test of 100 real live-streaming requests, the Harness-based system was preferred over a ReAct baseline in 97.2% of non-tied cases, primarily due to better input understanding and output reliability.
• Online A/B Test: A preliminary 7-day online test on Taobao Live showed a +5.54% increase in confirmed-receipt GMV per user and a +0.80% increase in completed orders per user compared to the ReAct control.

5. Significance and Claims

The paper claims that HAT successfully resolves the trade-off between model compactness (for latency) and adaptability (for business evolution). By treating the execution environment as a dynamic variable during training, the system produces an agent that can "evolve with its Harness."

The authors emphasize that this approach allows for agile business updates (changing strategies within hours via the Harness Evolution loop) without the prohibitive cost of model retraining or the performance degradation associated with fixed-Harness overfitting. The results suggest that for industrial agents operating in rapidly changing environments, the training distribution must explicitly reflect the variability of the runtime configuration. The work provides a practical blueprint for deploying robust, low-latency agents in high-stakes, real-time scenarios like live commerce.