Technical Summary: TMAX – A Simple Recipe for Terminal Agents

Problem Statement

Terminal-based agentic coding has become a dominant downstream application for Large Language Models (LLMs), with industry models increasingly expected to perform complex, long-horizon tasks via command-line interfaces. However, academic research in this domain faces three primary bottlenecks:

1. Data Scarcity: High-quality, open datasets for terminal agents are limited. Existing datasets often focus narrowly on bug-fixing (e.g., SWE-Bench subsets) or simple file manipulations, failing to cover the breadth of real-world terminal tasks.
2. Training Complexity: Reinforcement Learning (RL) for terminal agents is notoriously difficult to stabilize, particularly for long-horizon tasks, with many prior works relying on Supervised Fine-Tuning (SFT) alone or failing to demonstrate significant RL gains.
3. Lack of Baselines: There is a shortage of simple, reproducible "recipes" for training open-weight models to compete with closed-source offerings on complex terminal benchmarks like Terminal-Bench.

Methodology

The authors propose TMAX, a framework consisting of a novel data generation pipeline and a streamlined RL training recipe.

1. Data Generation: TMAX-15K

The authors introduce TMAX-15K, a dataset of 14,600 unique RL environment instances. Unlike prior works that rely on mining existing repositories or simple seed tasks, TMAX uses a compositional generation framework:

• Hierarchical Sampling: Tasks are constructed by sampling from nine structured axes, including Domain (e.g., security, data science), Skills, Task Complexity (shell commands to intricate workflows), Command Complexity (bash to system services), Persona, and Verifier types.
• Difficulty Control: The pipeline explicitly calibrates difficulty to avoid bi-modal distributions (trivial vs. unsolvable). It utilizes graded verifiers (metric thresholds, adversarial corpora, fuzz-equivalence) rather than simple binary correctness checks.
• Scalability & Diversity: The pipeline skips expensive teacher-based validation steps, relying instead on the RL infrastructure's "soft filtering" (discarding samples where the policy pass rate is zero). It generates diverse inputs, including multi-modal fixtures (images, audio, binaries) that agents must inspect via standard terminal tools (OCR, ffmpeg), rather than requiring a multi-modal model.
• Scale: The dataset is over 2.5x larger than the next largest terminal dataset releasing full environment data and demonstrates superior domain balance compared to prior works.

2. Training Recipe

The authors employ a simple, outcome-only RL recipe using Divergence Proximal Policy Optimization (DPPO), a variant of GRPO designed to mask tokens when inference and training logprobs deviate.

• Infrastructure: Training is fully asynchronous using open-instruct and vLLM for rollouts, with sandboxes managed via Podman or Apptainer.
• Stability Mechanisms:
  • FP32 LM Head: To minimize training-inference mismatch, the language model head is kept in FP32 precision, which the authors found crucial for models like Qwen 3.5.
  • Large Group Size: Using a group size of 32 (vs. standard 8) improves stability in long-horizon scenarios.
  • No Initial SFT: For Qwen 3.5 models, the authors found that existing SFT datasets often degraded performance; thus, they trained directly with RL.
• Evaluation: Models are evaluated on Terminal-Bench 2.0 and Terminal-Bench Lite, as well as general benchmarks like SWE-Bench Verified and AIME.

Key Contributions

1. TMAX-15K Dataset: The release of a 14,600-instance dataset with explicit control over difficulty and diversity, significantly larger and more balanced than prior terminal agent datasets.
2. State-of-the-Art Open Model: The training of TMAX-9B, a 9B parameter model that achieves 27% on Terminal-Bench 2.0. This outperforms prior open models of similar size and even larger open models (e.g., 32B variants), approaching the performance of closed-source models like Claude Haiku 4.5.
3. Reproducible RL Recipe: A simple, open-source recipe (data, code, checkpoints) that outperforms prior open RL baselines (e.g., Endless Terminals, OpenThinker-Agent) and demonstrates that RL can yield significant gains (>5 points) over SFT-only baselines.
4. Generalization Evidence: The authors demonstrate that the RL training generalizes across:
   • Tasks: Improved performance on SWE-Bench Verified (+5 points) and AIME.
   • Harnesses: Improved performance across different evaluation harnesses (OpenHands, mini-SWE-agent, Terminus-2), suggesting the model learns general terminal skills rather than "harness-fitting."
   • Model Families: Successful transfer to Qwen 3 8B and Qwen 3.6 27B.

Results

• Terminal-Bench 2.0: TMAX-9B achieves 27%, dominating the Pareto frontier for models under 32B parameters.
• Dataset Comparison: Training on TMAX-15K yields significantly higher performance (57.2% on TB Lite, 28.8% on TB 2.1) compared to training on other datasets like Endless Terminals (52.6%/25.5%) or SWE-Smith (47.2%/21.0%).
• Training Dynamics: Models trained on TMAX-15K utilize more steps and longer token sequences over time, indicating the learning of complex reasoning and tool-use patterns.
• Stability: The use of DPPO and FP32 heads successfully mitigated training collapse, which was observed when using standard GRPO or smaller group sizes.

Significance and Claims

The paper claims that TMAX provides a strong baseline for future academic research on terminal agents. The authors argue that their work demonstrates that:

1. Simple recipes work: Complex, multi-stage generation pipelines are not strictly necessary; a simple compositional approach with explicit difficulty control is sufficient to generate high-quality data.
2. RL is effective for terminal agents: Contrary to the view that RL is unstable or ineffective for this domain, the authors show that with the right data and stability mechanisms (DPPO, FP32 heads), RL can significantly boost performance on complex, long-horizon tasks.
3. Generalization is possible: The training does not merely overfit to a specific harness or task distribution but imparts transferable skills for using terminal tools to solve diverse problems.

The authors acknowledge limitations, noting that their dataset is entirely synthetic and relies on a strong generator model. They also highlight that training stability remains a challenge, with runs often collapsing after 300 steps, suggesting that further improvements in infrastructure and training algorithms could yield even better results. They position their work as a foundation for the community to build upon, rather than a final solution to terminal agent training.