💻 computer science

Nautilus: From One Prompt to Plug-and-Play Robot Learning

The paper introduces NAUTILUS, an open-source harness that transforms single prompts into automated, validated robot learning workflows by providing plug-and-play agent skills, unified interfaces, and scalable chambered execution to eliminate the engineering fragmentation currently hindering cross-family policy reproduction and evaluation.

Original authors: Yufeng Jin, Jianfei Guo, Xiaogang Jia, Yu Deng, Zechu Li, Han Liu, Weiran Liao, Vignesh Prasad, Mathias Franzius, Gerhard Neumann, Georgia Chalvatzaki

Published 2026-05-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yufeng Jin, Jianfei Guo, Xiaogang Jia, Yu Deng, Zechu Li, Han Liu, Weiran Liao, Vignesh Prasad, Mathias Franzius, Gerhard Neumann, Georgia Chalvatzaki

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the world of robot learning as a massive, chaotic construction site. On one side, you have architects designing new robot brains (called "policies"). On another side, you have test tracks (benchmarks) where robots learn to walk or pick up objects. On a third side, you have the actual robots themselves, ranging from simple arms to human-like walkers.

The problem, according to this paper, is that connecting any one architect to any one test track to any one robot is a nightmare. Every time a researcher wants to test a new brain on a new track with a new robot, they have to build a custom bridge from scratch. They have to write "glue code" to make the languages match, fix conflicting software versions, and ensure the robot doesn't crash. It's like trying to plug a toaster into a car engine, a toaster into a spaceship, and a toaster into a boat—each time, you have to invent a new adapter.

Enter Nautilus.

The authors propose Nautilus as a universal "plug-and-play" harness. Think of it as a smart, magical adapter station for the entire robot learning industry.

Here is how it works, using simple analogies:

1. The "Universal Power Strip" (Typed Contracts)

Currently, every robot and every brain speaks a different language. Nautilus introduces a standardized power strip.

The Analogy: Imagine if every appliance in your house (toaster, lamp, TV) had to be rewired to fit the wall outlet. Nautilus says, "No more rewiring." It forces every robot brain and every test track to agree on a single, strict shape for their plugs (called "typed contracts").
The Result: Once a robot brain is plugged into this standard, it can instantly connect to any test track or robot without needing a custom adapter.

2. The "Soundproof Booths" (Chambered Execution)

Robot software is messy. One robot might need a specific version of a video game engine, while another needs a different version of a physics simulator. If you run them on the same computer, they fight and crash.

The Analogy: Nautilus builds soundproof booths (containers) for each part of the experiment. The robot brain lives in one booth, the test track in another, and the robot itself in a third. They can talk to each other through a secure window, but their messy internal tools never clash.
The Result: You can mix and match any combination without worrying about software conflicts.

3. The "Smart Project Manager" (The Agent)

The paper uses a Large Language Model (an AI coder) to do the heavy lifting. But usually, AI coders fail at robotics because they don't know the "unwritten rules" of the field.

The Analogy: Nautilus gives the AI coder a rulebook and a checklist (called Guides and Sensors).
- Guides: Before the AI writes code, the rulebook tells it, "Hey, robots need to reset before starting, and you must check if the battery is safe."
- Sensors: After the AI writes code, a safety inspector checks it. "Did you actually test if the robot moves? Did you check the data format?" If the answer is no, the AI has to fix it before proceeding.
The Result: The AI doesn't just guess; it follows a proven, safe workflow that researchers use every day.

4. The "Nautilus Shell" (Scalability)

The name comes from the nautilus shell, which grows by adding new chambers.

The Analogy: Instead of building a giant, rigid framework that breaks when you add a new robot, Nautilus grows by adding new chambers. If a researcher adds a new robot (like a new type of hand), they just add one new "chamber" to the system. The rest of the system doesn't need to be rebuilt.
The Result: The system gets bigger and more useful without getting more complicated.

What Did They Actually Prove?

The paper doesn't claim that Nautilus invented a new robot brain or that it made robots smarter than they were before. Instead, they proved:

It works as a translator: They took existing robot brains and tested them on existing benchmarks using Nautilus, and the results matched what the original authors claimed.
It saves time: They showed that using Nautilus to connect a new robot brain to a new test track is much faster and cheaper than doing it manually. It reduces the work from building a bridge for every single pair to just building one bridge for the whole system.
It works on real robots: They successfully took a robot brain trained in a simulation, wrapped it in Nautilus, and deployed it on two very different real robots (a Franka robotic arm and a Unitree H1 humanoid robot) without changing the brain's code.

In short: Nautilus isn't a new robot brain; it's the universal translator and safety inspector that lets different robot brains, test tracks, and robots finally talk to each other without needing a team of engineers to build a custom bridge every single time.

Technical Summary: Nautilus: From One Prompt to Plug-and-Play Robot Learning

1. Problem Statement

Robot learning research is currently fragmented across three largely independent axes: policy families ( $P$ , e.g., IL, RL, VLA, WAM), benchmark suites ( $B$ , e.g., LIBERO, ManiSkill), and robot embodiments ( $R$ , e.g., Franka, Unitree H1). Integrating any specific triplet $(P, B, R)$ typically requires a distinct, hand-written integration layer involving Docker setup, observation adapters, and trust-validation procedures.

This fragmentation creates a "harness gap." While Large Language Models (LLMs) have advanced in code generation, they frequently fail to produce end-to-end reproduced results in robotics because they lack the procedural priors and validation practices specific to robotics research workflows. The engineering burden scales as $\Theta(N \cdot M \cdot K)$ , where $N, M, K$ are the cardinalities of policies, benchmarks, and robots. Existing frameworks (e.g., LeRobot, Isaac Lab) are often coupled to fixed components, while general-purpose coding agents lack the domain-specific scaffolding required to navigate complex dependency conflicts and interface mismatches.

2. Methodology: The NAUTILUS Harness

The authors propose NAUTILUS, an open-source, agent-facing harness designed to turn a single natural language prompt (e.g., "Evaluate policy A with benchmark B") into ready-to-use reproduction, evaluation, fine-tuning, and deployment workflows. NAUTILUS is structured into two layers: a substrate and a content layer.

2.1 The Substrate (Engineering Invariants)

The substrate provides a stable surface for the agent to reason over, reducing integration complexity from $\Theta(N \cdot M \cdot K)$ to $\Theta(N + M + K)$ . It relies on three invariants:

Typed Interface Contracts: Policies, benchmarks, and robots are exposed through fixed, machine-readable schemas (e.g., infer(obs) -> actions, reset(seed), apply_action). These contracts hide implementation-specific machinery (e.g., CUDA stacks, ROS 2 drivers) and define the unit of integration.
Chambered Execution: Generated artifacts run in disposable, isolated containers. This prevents dependency failures (e.g., conflicting CUDA versions) from leaking across modules. A policy container, a benchmark container, and a hardware container occupy the same architectural slot from the agent's perspective.
Uniform Inter-Module Communication:
- Tier 1 (Robot-local): High-bandwidth sensing and control remain in the native robot stack (e.g., ROS 2, shared memory).
- Tier 2 (Off-robot): The agent only sees a lightweight WebSocket protocol connecting the policy container to the benchmark or real-robot endpoint. This ensures simulated and real-robot endpoints share a single observation/action schema.

2.2 The Content Layer (Guides, Sensors, State)

Built atop the substrate, this layer distills robotics research workflows into explicit, reusable components:

Guides: Pre-generation instructions that steer the agent's behavior, providing routing rules, subagent roles, and validation protocols before code is written.
Sensors: Post-generation validators that check artifacts against six criteria (e.g., pre-action filtering, interface verification, spec comparison, smoke testing). Sensors can trigger local corrections or block runs if contracts are violated.
State: A persistent, JSON-schema-validated registry that stores typed contracts and pinned artifacts (source commits, Docker images). This allows the agent to reuse verified module knowledge rather than rediscovering repository layouts from scratch.
Workspace Artefacts: Append-only logs and reproduction receipts that record the source commit, environment choices, and decisions for every run.

2.3 Execution Flow

When a user issues a prompt (e.g., "Run $\pi_0$ on LIBERO"), the orchestrator routes the request to a specific subagent workflow. The system checks the State registry for verified entries. If missing, it generates policy and benchmark modules in parallel, compares their typed specs (e.g., observation keys, action shapes), executes a WebSocket smoke test, and writes reproducibility receipts.

3. Key Contributions

Modular Substrate: A system based on typed contracts, chambered execution, and uniform transport that collapses the integration cost of cross-family robot learning.
Distilled Procedural Priors: The conversion of tacit robotics workflow knowledge into explicit Guides and Sensors, turning the agent's "black box" generation into a validated, step-by-step process.
Open-Source Platform: The release of NAUTILUS with structured workflows supporting four policy families (IL, VLA, RL, WAM), six benchmark suites, and four embodiment classes. It includes nautilus-collect, a data-collection platform that validates the same Robot contract across heterogeneous hardware (Franka and Unitree H1).
Extensible Registry: A community platform where new policies, benchmarks, or robots can be onboarded via 3–5 typed method contracts without modifying framework code, provided they pass trust-validation protocols.

4. Experimental Results

The paper evaluates NAUTILUS through three studies:

Benchmark Trust Validation: The system reproduced reference results for public policies (Diffusion Policy, $\pi_0$ , OpenVLA, etc.) across LIBERO, ManiSkill, RoboTwin, ALOHA, and RoboCasa. The generated wrappers faithfully reproduced reference scores with small deviations (e.g., $\Delta \approx \pm 2\%$ ), confirming that the adapters preserve intended evaluation protocols.
Ablation on Extensibility: In a "leave-one-benchmark-out" testbed, NAUTILUS was compared against a vanilla coding agent.
- Success: NAUTILUS achieved 100% wrap success and high task success (98.7% on Claude Opus 4.7).
- Cost Efficiency: Removing the typed template (the substrate) caused wrap success to drop to 55.3% and token usage to triple (41.7M tokens), demonstrating that the substrate prevents the agent from relearning glue code.
- Model Scaling: The harness provided greater relative benefits on weaker models (Claude Sonnet 4.6), reducing interaction turns by ~27% and costs by ~8% compared to the vanilla agent.
Real-Robot Deployment: The system successfully deployed a VLA manipulation checkpoint ( $\pi_0.5$ ) on a Franka arm and an RL locomotion policy on a Unitree H1. In both cases, the same policy artifact wrapped for simulation was transferred to hardware by swapping only the hardware container, with no changes to the policy code or platform-side container.

5. Significance and Claims

The paper argues that the primary constraint on agentic automation in robot learning is not the capability of the LLM model, but the harness engineering surrounding it. NAUTILUS demonstrates that by providing a stable, contract-based substrate with explicit validation, the engineering burden of cross-family reproduction and evaluation can be drastically reduced.

The authors claim that NAUTILUS transforms fragmented research workflows into structured, agentic procedures. By making domain expertise (contracts, validation protocols, state management) executable, reusable, and inspectable, the system enables researchers to scale their workflows without hand-curating glue code for every new $(P, B, R)$ combination. The work positions NAUTILUS not just as a tool for automation, but as a step toward a future where expert-built harnesses allow models to scale domain expertise across the entire robot learning ecosystem.