🤖 AI

SETA: Scaling Environments for Terminal Agents

The paper introduces SETA, a scalable framework for generating verifiable terminal environments that produces the largest open-source dataset of its kind (SETA-Env), enabling RL-trained models like Qwen3-8B to achieve state-of-the-art performance on terminal agent benchmarks.

Original authors: Qijia Shen, Zhiqi Huang, Vamsidhar Kamanuru, Aznaur Aliev, Jay Rainton, Ahmed Awelkair, Zhichen Zeng, Jiajun Li, Shi Dong, Yueming Yuan, Boyuan Ma, Qizheng Zhang, Jiwei Fu, Yuzhen Mao, Wendong Fan, Pi

Published 2026-07-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Qijia Shen, Zhiqi Huang, Vamsidhar Kamanuru, Aznaur Aliev, Jay Rainton, Ahmed Awelkair, Zhichen Zeng, Jiajun Li, Shi Dong, Yueming Yuan, Boyuan Ma, Qizheng Zhang, Jiwei Fu, Yuzhen Mao, Wendong Fan, Ping Nie, Philip Torr, Bernard Ghanem, Changran Hu, Jonathan Lingjie Li, Urmish Thakker, Guohao Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you're trying to teach a super-smart robot how to use a computer's command line (that text-only screen where you type commands like ls or cd). It's a bit like teaching a wizard to cast spells by only letting them whisper to a very literal, very strict genie. The genie (the computer) does exactly what you say, but if you miss a tiny detail, the spell fizzles, and the room goes dark.

For a long time, teaching these "terminal agents" has been a nightmare. Why? Because you can't just ask a human to write down every possible command they've ever typed. And if you ask a robot to make up its own practice problems, it often creates puzzles that are impossible to solve or don't actually teach anything useful. It's like trying to learn to ride a bike by reading a book written by someone who has never touched a bicycle.

Enter SETA (Scaling Environments for Terminal Agents). Think of SETA as a magical, self-updating "training gym" for these robot wizards. Instead of just giving them a static list of drills, SETA builds a whole world where they can practice, fail, and learn in real-time.

The Two-Step Magic Trick

The researchers built SETA using two clever pipelines that work together like a master chef and a sous-chef:

The "Chef" (SETA-Synth): This part goes out and grabs real-world recipes from the internet. It looks at places where real humans solve problems, like Stack Overflow (where people ask "How do I fix this broken code?") or Kaggle (where data scientists share their workbooks). It takes these real-life puzzles and turns them into a standardized "training environment." It's like taking a messy, real-world kitchen disaster and turning it into a clean, safe practice kitchen with a clear goal: "Fix the stove so the soup doesn't burn."
- Crucial Check: The system doesn't just trust the robot chef. It runs a "no-op" test (does nothing and checks if the kitchen is still safe) and an "oracle" test (does the perfect solution work?). If the robot's solution fails, a special "Trajectory Judge" steps in. This judge is like a strict referee who looks at why the robot failed. If the robot failed because the puzzle was broken (a design flaw), the puzzle gets thrown in the trash. If it failed because the robot just needs more practice, the puzzle stays.
The "Sous-Chef" (SETA-Evol): This is the adaptive part. Imagine you're training a robot. If the puzzles are too easy, the robot gets bored and learns nothing. If they're too hard, the robot gives up. SETA-Evol watches how the robot is doing.
- If the robot is crushing a task (solving it 100% of the time), the system makes it harder by adding more steps or tricky edge cases.
- If the robot is stuck and failing, the system makes it easier by removing a step or giving a hint.
- If the robot is doing okay but needs variety, the system swaps the context. Maybe it was fixing a Linux server; now, it's fixing a Windows server, but the logic is the same. This keeps the robot sharp and adaptable.

The Results: A Robot That Actually Learns

The team put this system to the test. They took a model called Qwen3-8B (a robot brain with 8 billion "neurons") and trained it using this new gym.

The Score: Before training, the robot could only solve about 3.6% of the tough terminal challenges on a standard test called Terminal-Bench 2.0. After training with SETA, it jumped to 12%. That's a 3.3 times improvement!
The Comparison: This result is the best anyone has reported for a robot of this size (8 billion parameters) using Reinforcement Learning (learning by trial and error). It even came close to much bigger, proprietary models.
The Transfer: They also tried this on a different, stronger robot called DeepSeek-V4-Flash. Even though this robot was already pretty good, the SETA training still gave it a boost, improving its success rate from 40% to 43%. This suggests the training isn't just a trick for one specific robot; it's a genuine way to teach terminal skills.

What SETA Is Not

It's important to know what this paper doesn't claim. The authors are very careful not to say they've "solved" terminal agents.

They didn't say this works for every robot. They tested it on a few specific ones (Qwen and DeepSeek).
They didn't say this works for all computer interfaces. This is specifically for the text-based command line, not for clicking icons on a screen (GUI) or using a mouse.
They didn't claim the data is perfect. They explicitly mention that they had to filter out "broken" puzzles and that the system relies on the robot's ability to verify its own work.

The Takeaway

The paper suggests that if you want to teach a robot to be a computer wizard, you can't just give it a textbook. You have to build it a gym where the exercises get harder as it gets stronger, and where every exercise is checked to make sure it's actually solvable.

By combining real-world human problems with a system that constantly adjusts the difficulty, SETA created a dataset of over 4,500 verified training environments. It's not a magic wand that makes robots perfect overnight, but it's a powerful new tool that suggests we can scale up robot learning in a way that actually works. The robot didn't just memorize answers; it learned how to think through the messy, unpredictable reality of a computer terminal.

Technical Summary: SETA: Scaling Environments for Terminal Agents

Problem Statement

Large Language Models (LLMs) are increasingly deployed as agents that interact with diverse interfaces, including web and graphical user interfaces (GUIs). Among these, the terminal command line offers a text-based, general-purpose interface covering system operations, data science, and machine learning tasks. However, scaling training for terminal agents remains a significant bottleneck. Unlike software engineering agents that rely on natural supervision signals like GitHub pull requests, terminal tasks lack grounded supervision. Furthermore, existing approaches struggle to simultaneously synthesize diverse task instructions, executable environments, and reliable verification logic. Crucially, current methods often lack adaptive difficulty control, resulting in data distributions skewed toward the synthesis source rather than the "productive RL zone" where tasks are challenging yet learnable for the specific model.

Methodology

The authors propose SETA (Scaling Environments for Terminal Agents), a scalable framework designed to generate verifiable terminal environments for Reinforcement Learning (RL). The framework operates through two consecutive pipelines sharing a unified verification mechanism:

1. SETA-Synth: Source-Adaptive Synthesis

This pipeline converts diverse, human-validated sources into standardized, verifiable terminal environments.

Sources: It aggregates data from community Q&A platforms (Stack Overflow, Unix & Linux StackExchange), data science notebooks (Kaggle), and curated command datasets (NL2Bash).
Architecture: It employs a source-adaptive architecture using an Idea Agent to convert raw source examples into a unified intermediate draft specification. This agent uses source-specific adapters to handle format variations while maintaining a common task-design framework.
Instantiation: A Datapoint Agent then instantiates these drafts into complete environments, generating Dockerfiles, ground-truth solution scripts (solve.sh), test suites (test_state.py, test.sh), and task instructions.
Verification: The pipeline enforces a SWE-bench-style verification protocol:
- No-op Test: Ensures the environment fails without agent intervention.
- Oracle Test: Confirms the ground-truth solution passes all tests.
- Post-Rollout Verification: A separate Trajectory Judge Agent audits tasks with 100% rollout failure rates to distinguish between genuine difficulty and systematic design flaws (e.g., unstated conventions in tests), discarding the latter to prevent dataset pollution.

2. SETA-Evol: Adaptive Evolution

This pipeline expands the dataset by adaptively reshaping the difficulty and category distribution based on the training model's performance.

Strategy Selection: Based on the model's pass rate ( $r$ $r$ ) on a specific task, SETA-Evol selects an evolution operator:
- Difficulty Increase ( $d_1$ ): For easy tasks ( $r > 0.5$ ), it adds constraints, edge cases, or multi-step dependencies to push tasks toward the productive frontier.
- Context Shift ( $b_1$ ): For medium-difficulty tasks ( $r \in (0, 0.5]$ ), it changes the domain or toolset (e.g., swapping sklearn for caret, or systemd for init.d) while preserving difficulty to increase diversity.
- Difficulty Decrease ( $d_1$ ): For difficult tasks ( $r \approx 0$ ), it simplifies the task by removing steps or edge cases to make it learnable.
Iterative Process: Evolved tasks undergo the same rigorous validation pipeline as synthesized ones, ensuring continuous scaling of high-quality data.

Key Contributions

SETA-Synth Pipeline: A novel synthesis method that grounds terminal environment generation in diverse, human-validated sources using per-source adapters and a post-rollout trajectory judge to catch flaws missed by oracle verification.
SETA-Evol Framework: A difficulty-adaptive evolution system that calibrates task difficulty to the training model's capability frontier, shifting distributions toward the productive RL zone and expanding contextual diversity.
SETA-Env Dataset: The release of SETA-Env, the largest open-source verifiable terminal RL dataset to date, containing 4,567 environments across 14 categories (e.g., system administration, machine learning, data processing).
State-of-the-Art Results: Demonstration of significant performance gains on terminal benchmarks using RL-trained models.

Experimental Results

The authors evaluated the framework by training Qwen3-8B and DeepSeek-V4-Flash using Group Relative Policy Optimization (GRPO) on SETA-Env.

Terminal-Bench 2.0 Performance:
- Qwen3-8B: The RL-trained model achieved a 12% pass@1 on Terminal-Bench 2.0, a 3.3x improvement over the base model (3.6%). This represents the best reported result for an 8B-scale RL-trained model, approaching the performance of the 120B-scale GPT-OSS-120B.
- DeepSeek-V4-Flash: Training on SETA-Env improved pass@1 from 40% to 43% and pass@5 from 54% to 58%, demonstrating that the training signal transfers across different model families.
Generalization: The model showed gains on adjacent coding benchmarks, including CompileBench (6.7% $\to$ 40%) and CRUST-Bench (15% $\to$ 24%), indicating that terminal capabilities generalize to iterative compilation and code translation tasks.
Behavioral Shift: Analysis revealed that RL-trained models exhibit structured reasoning and planning before executing commands, adapting to concrete execution errors (e.g., missing sudo in Docker) rather than reacting blindly.

Significance and Claims

The paper claims that SETA provides a practical and scalable pipeline for generating training environments and learning signals for terminal agents. By integrating executable verification, adaptive difficulty control, and diverse source grounding, SETA-Env addresses the critical lack of high-quality training data for terminal agents.

The authors emphasize that their results demonstrate that scalable environment synthesis, combined with strong verification, can effectively support RL training for agentic systems in interactive settings. They position SETA-Env as a valuable resource for advancing research in terminal-based agent learning, noting that the dataset is not yet saturated by strong proprietary models, preserving its utility for future research.

Limitations: The authors acknowledge that their validation is currently limited to two model families (Qwen and DeepSeek) and specific scales. They also note that the environments are restricted to terminal-based interactions, leaving GUI and multimodal extensions as directions for future work. Additionally, they highlight the potential for misuse (unauthorized system access) and state that they mitigate this by deriving tasks from public, non-sensitive sources and emphasizing sandboxed, verifiable environments.