Technical Summary: Agentick: A Unified Benchmark for General Sequential Decision-Making Agents

Problem Statement

Research in autonomous agents spans a wide spectrum, ranging from deep reinforcement learning (RL) agents that learn from scratch to foundation model (FM) agents (LLMs and VLMs) that leverage pre-trained world knowledge. Despite this diversity, a critical gap exists: no unified benchmark enables fair comparison across these paradigms. Existing benchmarks are typically designed for specific agent types (e.g., pixel-based environments for RL or text-based games for LLMs), making it impossible to evaluate whether a combination of interaction-based learning and pre-trained knowledge is necessary for building fully capable agents. Furthermore, current benchmarks often lack the infrastructure to support training, fail to decompose capabilities into specific dimensions, or do not offer standardized observation modalities that prevent architectural bias.

Methodology: The Agentick Benchmark

The authors present Agentick, a benchmark designed from the ground up to support fair evaluation across the full agent design spectrum (RL, LLM, VLM, hybrid, and human agents). The benchmark is built on four guiding principles:

1. Paradigm Universality: Every task exposes five synchronized observation modalities simultaneously:

   • ASCII text grids (for LLMs).
   • Natural language descriptions.
   • Structured dictionaries.
   • 512×512 isometric pixel renderings (for VLMs and CNN-based RL).
   • Raw numpy state arrays (for programmatic agents).
     This ensures no agent type is disadvantaged by the input format.

2. Capability Decomposition: Instead of a single aggregate score, evaluation is broken down across six capability categories:

   • Navigation (8 tasks)
   • Planning (9 tasks)
   • Reasoning (8 tasks)
   • Memory (4 tasks)
   • Generalization (3 tasks)
   • Multi-Agent Coordination (5 tasks)
     This allows for "radar-chart" profiling to identify specific strengths and weaknesses.

3. Training-First Design: The benchmark includes a Coding API for programmatic task interaction, oracle reference policies for all 37 tasks, and pre-built Supervised Fine-Tuning (SFT) datasets (120k–500k oracle episodes) available on HuggingFace. Environments are vectorizable for parallel RL training.

4. Controlled Difficulty: Tasks feature four difficulty levels (easy to expert) and are procedurally generated to ensure unique layouts at every seed, scaling grid size, constraint complexity, and episode length.

Evaluation Metric: The primary metric is the Oracle-Normalized Score (ONS):
$$ONS = \frac{\text{agent\_return} - \text{random\_baseline}}{\text{oracle\_return} - \text{random\_baseline}}$$
This normalizes performance relative to a random baseline and a strong reference policy (oracle), enabling cross-task comparison.

Experimental Setup: The authors evaluated 27 configurations across 37 tasks, 4 difficulty levels, and 25 seeds per task-difficulty pair (totaling >90,000 episodes). The evaluated agents included:

• Frontier LLMs: GPT-5 mini, Gemini 3.1 Flash Lite, Claude Haiku 4.5.
• RL from Scratch: PPO agents trained with dense and sparse rewards.
• Open-Weight LLMs: Qwen3 and Qwen3.5 models (0.8B to 4B parameters).
• Harnesses: Agents were tested with different inference strategies, including a "Markovian Reasoner" harness that prompts for chain-of-thought reasoning.

Key Results

The evaluation reveals that no single paradigm dominates across all capabilities:

• Overall Performance: GPT-5 mini achieved the highest overall ONS (0.309), followed closely by PPO trained from scratch (0.287). However, even the best agent reaches less than one-third of the oracle performance, indicating substantial room for improvement.
• Capability Specialization:
  • GPT-5 mini excelled in Navigation (0.456) and Generalization (0.437), leveraging pre-trained knowledge.
  • PPO dominated Planning (0.402) and Multi-Agent tasks (0.432), where precise, repeatable control learned through interaction is critical.
  • Reasoning tasks remained the weakest category for all agents, with the best score (PPO) at only 0.191.
• Impact of Harness Design: The choice of prompting strategy was as critical as model scale. The Reasoner harness (chain-of-thought) multiplied LLM performance by 3–10× across all tested models. For example, Qwen3.5-4B jumped from 0.023 to 0.228 ONS when switching from a standard Markovian harness to the Reasoner harness.
• Observation Modality: ASCII observations consistently outperformed natural language for LLM agents (e.g., Qwen3.5-4B scored 0.228 on ASCII vs. 0.181 on language), suggesting compact, token-efficient representations are superior for spatial reasoning.
• Model Efficiency: Smaller models with better harnesses outperformed larger models with weaker setups (e.g., Qwen3.5-0.8B with Reasoner outperformed Qwen3-4B without it).

Significance and Claims

The authors claim that Agentick provides the empirical infrastructure necessary to drive progress toward general autonomous agents. Its significance lies in:

1. Enabling Cross-Paradigm Comparison: It is the first benchmark to support fair evaluation of RL, LLM, VLM, and hybrid agents on the same tasks with standardized metrics.
2. Driving RL Post-Training: It serves as a training ground for RL post-training of foundation models in truly sequential, stochastic, and interactive environments, extending beyond current RLVR (Reinforcement Learning from Verifiable Rewards) successes in static domains like math and code.
3. Revealing Fundamental Gaps: The results highlight that current approaches are insufficient for complex sequential decision-making, particularly in reasoning and long-horizon planning, suggesting that hybrid architectures combining FM reasoning with RL-trained control may be required.
4. Research Utility: By providing a composable harness, oracle datasets, and a live leaderboard, Agentick facilitates the study of agent scaffolding, curriculum learning, and the effects of observation modalities on agent performance.

The paper concludes that while no single approach currently dominates, the benchmark successfully discriminates between paradigms and highlights the substantial research agenda ahead for building truly general autonomous agents.