Technical Summary: Interactive Training 2 – Auditable Control Plane for Live Model Training

Problem Statement

While modern experiment trackers (e.g., Aim, Weights & Biases) effectively visualize training progress, they lack a general mechanism for executing interventions on live training runs. Currently, modifying a running experiment—such as adjusting learning rates, rebalancing data, or saving checkpoints—requires trainer-specific code, custom callbacks, or manual logic tailored to a single framework. This fragmentation prevents researchers from easily applying human or automated steering across different training workflows. Although researchers frequently make decisions based on live behavior, the tooling to act on these observations remains disjointed and non-reusable.

Methodology

The authors propose Interactive Training 2 (IT2), an open-source control plane that decouples training logic from control logic through a shared protocol. The system operates on three core principles:

1. The Control Plane Architecture

The system introduces a layer between the training code and its controllers (humans, scripts, or LLM agents).

• Declaration: Training applications explicitly declare which settings (e.g., learning rate, source weights) and actions (e.g., evaluation, checkpointing) are exposed. These are registered as typed "knobs" and structured actions via a Python API (register_knob).
• Submission: Controllers submit requests through a unified interface (HTTP API or WebSocket). All controllers, regardless of type, use the same request format.
• Execution: The training loop retains full control. It processes requests at safe control points (explicitly defined by the developer) rather than immediately upon receipt. At these points, the system validates requests, applies changes, and records outcomes before proceeding to the next training step.

2. The Auditable Journal

A customized Aim workspace serves as the central interface, combining live metrics with a chronological, ordered journal. This journal links:

• Controller decisions (plans).
• Submitted requests and their sources.
• Execution results (success/failure).
• Evaluation metrics and checkpoints.
• Reflections (in the case of LLM agents).

This structure allows for the reconstruction of the entire training history: who requested what, when it was applied, and the resulting impact on the model.

3. Integration Mechanisms

The system supports three methods for connecting training code to the control plane:

• Trainer Callbacks: For Hugging Face Trainer, a callback registers optimizer controls and maps session decisions to trainer flags.
• Optimizer Patching: An optional wrapper turns optimizer.step() into a control point for loops difficult to edit.
• Direct Integration: Custom PyTorch loops and Reinforcement Learning with Verifiable Rewards (RLVR) loops call session.step directly, allowing full control over diagnostics and checkpointing logic.

4. Optional LLM Agent

The system includes an optional LLM agent that operates in three phases:

• Plan: Generates an initial configuration and strategy based on task goals and previous rounds.
• Act: Issues tool calls (e.g., changing hyperparameters) based on current metrics and registered actions.
• Reflect: Analyzes the completed trajectory and writes actionable lessons for the next round.
  Crucially, the LLM agent uses the same protocol as human users, and each new round starts with a fresh model and optimizer, passing only a text summary of configurations and reflections to the next iteration.

Key Contributions

The paper outlines three primary contributions:

1. A Shared Protocol for Steering: A reusable interface where applications expose typed settings and actions, while training loops determine safe application points. This moves beyond trainer-specific implementations to a general standard.
2. A Unified Interface for Observation and Control: A customized Aim workspace that integrates live metrics with an ordered journal, connecting controller decisions to execution results and evaluations in a single view.
3. Open-Source Implementation: A demonstrated system across five diverse workflows (NLP and RL), supporting callbacks, optimizer wrappers, and direct loops. The code is available for humans, scripts, and agents to use interchangeably.

Results and Validation

The system was validated across five workflows, demonstrating the protocol's flexibility and the system's ability to produce auditable records:

• Workflows Tested:
  • Sentiment classification (Hugging Face Trainer).
  • Sentiment mixing (Direct PyTorch loop).
  • Layerwise GPT (Direct PyTorch loop with 27 distinct learning rates).
  • Muon–AdamW GPT (Direct PyTorch loop with optimizer diagnostics).
  • RLVR Countdown (Direct RL loop with curriculum controls).
• Performance: In all cases, the system successfully recorded the request-to-result path. For example, in the Sentiment mixing task, the macro-F1 score improved from 0.568 to 0.656 over 11 rounds. In the RLVR task, the hardest accuracy improved from 0.032 to 0.232.
• Audibility: The system successfully demonstrated that later rounds could visibly utilize reflections from earlier rounds. For instance, after a regression in Round 8 of the sentiment task, the reflection proposed specific weight adjustments, which were explicitly recorded and applied in Rounds 9 and 10, leading to score improvements.
• Traceability: The released JSONL journals allow for the verification of every action, from the initial request to the final checkpoint, ensuring that the sequence of decisions is reconstructible.

Significance and Claims

The paper positions Interactive Training 2 as a shift from experiment-specific logic to a reusable interface for live training intervention.

• Modularity: By separating the definition of what can change and when it applies from the who (controller), the system enables researchers to add human or automated steering to different workflows without rebuilding monitoring and logging machinery for each experiment.
• Auditability: The system claims to provide a foundation for "auditable human- and agent-guided training," allowing for the systematic study of how decisions are made and their effects on learning progress.
• Limitations: The authors modestly note that the training loop waits for LLM responses (blocking), the system does not yet handle late-arriving responses, and safety guarantees (e.g., preventing catastrophic hyperparameter changes) rely on application-level checks, value bounds, and permissions rather than being inherent to the protocol itself. High-stakes use requires additional approval gates and rollback policies.

In summary, Interactive Training 2 provides a standardized, auditable control plane that makes live training runs steerable, reusable, and inspectable across diverse machine learning frameworks.