Bittensor Agent Arenas as a Trajectory Primitive: Distilling a Shopping Agent from ShoppingBench Subnet Traces

Technical Summary: Bittensor Agent Arenas as a Trajectory Primitive

Problem Statement
The paper identifies a critical bottleneck in small-model agentic post-training: the scarcity of high-quality, per-trajectory supervised data. While leading algorithms (RLVR, group-relative RL, rejection-sampled re-SFT) demonstrate measurable performance lifts, they rely on multi-turn traces with independent supervision that existing data sources fail to provide at scale.

• Synthetic Data: Generated by frontier models, this data inherits the synthesizer's biases and causes distribution collapse (model collapse), failing to capture the long tail of diverse agent behaviors.
• Production Logs: While diverse, unfiltered logs lack per-trajectory judgment, are dominated by the current top policy, and are contaminated by "shortcut" behaviors where agents reach correct answers via illegitimate reasoning.

The authors argue that small-model agentic post-training requires a trajectory substrate that offers incentive-aligned diversity, per-trajectory judging, and anti-memorized held-out evaluation—properties not found in current isolated data sources.

Methodology
The authors propose and demonstrate a novel data generation primitive: an incentive-aligned agent arena built on the Bittensor network. They instantiate this via ORO Subnet 15 (SN15), a deployment of the ShoppingBench agentic-commerce benchmark.

1. Arena Mechanics (SN15):

   • Incentive-Aligned Diversity: Independent miners submit AI agents to a continuous race. A daily promotion cycle, submission rate limits, and source code embargoes create a "discover-then-share" pressure, ensuring diverse policies are continuously generated to compete for token rewards.
   • Per-Trajectory Judging: Every trajectory is scored on two axes: an outcome score (rule-based) and a reasoning-quality coefficient assigned by a dedicated LLM reasoning judge. The judge is trace-grounded, decoupled from the outcome scorer, and run by validators, not miners.
   • Rotating Held-Out Problems: The problem suite is versioned and rotated. A "leak-cluster guard" groups paraphrased or attribute-reordered variants of the same underlying problem, ensuring that held-out evaluation partitions remain distinct from training data to prevent memorization.

2. Data Pipeline & Structural Filter:
   The raw "firehose" of SN15 trajectories is processed into a trainable corpus:

   • Normalization: Traces are converted to a canonical OpenAI tool-calls schema using the Trac framework.
   • Structural-Quality Filter: A multi-stage filter selects only agentic trajectories (where the LLM emits tool calls) and rejects sub-task trajectories (where the LLM acts merely as a classifier or narrator over a deterministic search loop).
   • Gating: Trajectories must pass a reasoning-coefficient gate (based on the LLM judge) and a format-validity gate.
   • Leak-Cluster Guarded Split: The dataset is split into training and evaluation pools such that no problem variants (leak clusters) appear in both, ensuring a true generalization surface.

3. Post-Training Recipe:
   The authors fine-tune Qwen3-4B on the filtered corpus using a five-stage pipeline closely matched to the published ShoppingBench SFT-then-GRPO baseline, with two documented substitutions:

   • Continued teacher-SFT replaces step-wise on-policy distillation.
   • KTO preference refinement is added as a stage.
   • The pipeline concludes with Dr. GRPO (a per-step teacher-grounded reward) to convert latent headroom into process improvement.

Key Contributions

1. New Training Primitive: The paper characterizes Bittensor agent subnets as a new shape of training-data primitive, defined by the combination of incentive-aligned diversity, per-trajectory judging, and rotating held-out problems.
2. Reproducible Arena Mechanics: Detailed documentation of SN15's race system, LLM reasoning judge, and problem rotation, producing a continuous stream of judged trajectories.
3. Structural-Quality Filter: A novel filter that converts raw subnet traces into a trainable SFT corpus by distinguishing between true agentic behavior and sub-task narration, released as open code.
4. Empirical Validation: Demonstration that a small open model (Qwen3-4B) post-trained on this subnet-generated corpus achieves significant performance gains on a strict, leak-cluster-guarded held-out partition.

Results
On a leak-cluster-guarded held-out partition scored with production-strict criteria:

• Baseline: The published Qwen3-4B base model achieved an Average Success Rate (ASR) of 18.0%.
• Distilled Model: The model post-trained on the SN15 corpus achieved an ASR of 42.7%.
• Gain: This represents a 24.7-point improvement, landing within single-problem noise of the published synthetic-data SFT-only baseline (43.6%).
• Efficiency: This gain was achieved using a training slice that represents only a small fraction of a single day of subnet output.
• Remaining Gap: The supervised stack left a significant gap between pass@1 (34.8%) and pass@8 (53.3%). The authors identify the "sub-task firehose" (currently filtered out) as the primary lever for closing the remaining gap to the published SFT+GRPO bar of 48.7%.

Significance and Claims
The paper claims that an incentive-aligned agent arena can be deliberately engineered to manufacture the specific type of trajectory data required for small-model agentic post-training. It posits that the combination of economic incentives, external reasoning judges, and rotating problem sets creates a data substrate qualitatively different from both synthetic data and production logs.

The authors are modest in their scope, explicitly stating:

• They do not claim subnet trajectories can universally replace synthetic data or production logs for all task surfaces.
• The resulting model is not a finished product.
• They acknowledge risks, including the skew toward sub-task trajectories in the raw data (which are filtered out), the sensitivity of results to harness formats, and the fact that reasoning judge supervision is correlational rather than "gold" standard.

The work serves as a proof-of-concept that a decentralized, incentive-aligned competition can generate high-quality, judged, multi-turn trajectories capable of distilling competent agents from small base models.